Max level shown:
12025-03-20T22:00:01.509ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e21e0885-3842-46dd-9094-2890efe99231 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22025-03-20T22:00:01.509ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e21e0885-3842-46dd-9094-2890efe99231 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
32025-03-20T22:00:01.512ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2716 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e21e0885-3842-46dd-9094-2890efe99231 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
42025-03-20T22:00:02.539ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8117b395-655e-4502-bb23-c9d2482cc4bb roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52025-03-20T22:00:02.540ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8117b395-655e-4502-bb23-c9d2482cc4bb resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
62025-03-20T22:00:02.542ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2588 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8117b395-655e-4502-bb23-c9d2482cc4bb response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
72025-03-20T22:00:03.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d5044cef-4627-4c4b-9df0-0ded837aa4d3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
82025-03-20T22:00:03.568ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d5044cef-4627-4c4b-9df0-0ded837aa4d3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
92025-03-20T22:00:03.570ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2486 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d5044cef-4627-4c4b-9df0-0ded837aa4d3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
102025-03-20T22:00:04.593ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d4fbceab-5097-4240-99d8-05ff02df91b2 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
112025-03-20T22:00:04.594ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d4fbceab-5097-4240-99d8-05ff02df91b2 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
122025-03-20T22:00:04.595ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2119 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d4fbceab-5097-4240-99d8-05ff02df91b2 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
132025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
142025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
152025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
162025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
172025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
182025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
192025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
202025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
212025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
222025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
232025-03-20T22:00:05.239ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-20 21:50:05.239860170 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
242025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
252025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
262025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
272025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
282025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
292025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
302025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
312025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
322025-03-20T22:00:05.239ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
332025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
342025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
352025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
362025-03-20T22:00:05.240ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
372025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
382025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
392025-03-20T22:00:05.240ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f
402025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
412025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.341\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f"}
422025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
432025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
442025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
452025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
462025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
472025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
482025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
492025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
502025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
512025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
522025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
532025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
542025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
552025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
562025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
572025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
582025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
592025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
602025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
612025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
622025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
632025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
642025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
652025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
662025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
672025-03-20T22:00:05.240ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
682025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
692025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
702025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
712025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
722025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
732025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
742025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
752025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
762025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
772025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
782025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
792025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
802025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
812025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
822025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
832025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
842025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
852025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
862025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
872025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
882025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
892025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
902025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
912025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
922025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
932025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
942025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
952025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
962025-03-20T22:00:05.240ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55400d62-c4af-483b-8223-1234d14cf0ad.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3549, tv_nsec: 45714949 } })
972025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
982025-03-20T22:00:05.240ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58723511-3cb5-4c26-bebf-16620d9cd641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("59acd018-3b3d-4cb6-bd0f-2dfaf2b8d965.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 3549, tv_nsec: 45725289 } })
992025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1002025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1012025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1022025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1032025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1042025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1052025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1062025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1072025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1082025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1092025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1102025-03-20T22:00:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1112025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1122025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1132025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
1142025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.077559ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
1152025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1162025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1172025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
1182025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.059018ms iteration = 10 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
1192025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1202025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1212025-03-20T22:00:05.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1222025-03-20T22:00:05.243ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1232025-03-20T22:00:05.243ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.214537ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1242025-03-20T22:00:05.243ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1252025-03-20T22:00:05.243ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1262025-03-20T22:00:05.243ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1272025-03-20T22:00:05.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1282025-03-20T22:00:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1292025-03-20T22:00:05.246ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1302025-03-20T22:00:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1312025-03-20T22:00:05.248ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1322025-03-20T22:00:05.248ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-20 21:50:05.239860170 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
1332025-03-20T22:00:05.248ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.170378ms iteration = 10 status = {"expiration":"2025-03-20T21:50:05.239860170Z","pruned":[]}
1342025-03-20T22:00:05.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1352025-03-20T22:00:05.249ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3549, tv_nsec: 54439802 } })
1362025-03-20T22:00:05.249ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
1372025-03-20T22:00:05.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1382025-03-20T22:00:05.251ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.821706ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
1392025-03-20T22:00:05.252ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3549, tv_nsec: 58048779 } })
1402025-03-20T22:00:05.253ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
1412025-03-20T22:00:05.253ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1422025-03-20T22:00:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1432025-03-20T22:00:05.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1442025-03-20T22:00:05.254ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 13.958473ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
1452025-03-20T22:00:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1462025-03-20T22:00:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1472025-03-20T22:00:05.256ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1482025-03-20T22:00:05.258ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1492025-03-20T22:00:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1502025-03-20T22:00:05.259ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1512025-03-20T22:00:05.259ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1522025-03-20T22:00:05.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1532025-03-20T22:00:05.260ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1542025-03-20T22:00:05.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1552025-03-20T22:00:05.262ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1562025-03-20T22:00:05.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1572025-03-20T22:00:05.266ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1582025-03-20T22:00:05.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1592025-03-20T22:00:05.266ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1602025-03-20T22:00:05.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1612025-03-20T22:00:05.269ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1622025-03-20T22:00:05.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1632025-03-20T22:00:05.269ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1642025-03-20T22:00:05.270ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1652025-03-20T22:00:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1662025-03-20T22:00:05.270ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1672025-03-20T22:00:05.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1682025-03-20T22:00:05.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1692025-03-20T22:00:05.273ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1702025-03-20T22:00:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1712025-03-20T22:00:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1722025-03-20T22:00:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1732025-03-20T22:00:05.282ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
1742025-03-20T22:00:05.282ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
1752025-03-20T22:00:05.282ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 42.493776ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
1762025-03-20T22:00:05.306ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1772025-03-20T22:00:05.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1782025-03-20T22:00:05.314ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 74.857832ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
1792025-03-20T22:00:05.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1802025-03-20T22:00:05.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1812025-03-20T22:00:05.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1822025-03-20T22:00:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1832025-03-20T22:00:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1842025-03-20T22:00:05.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1852025-03-20T22:00:05.317ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1862025-03-20T22:00:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1872025-03-20T22:00:05.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1882025-03-20T22:00:05.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1892025-03-20T22:00:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1902025-03-20T22:00:05.320ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1912025-03-20T22:00:05.320ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1922025-03-20T22:00:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1932025-03-20T22:00:05.321ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1942025-03-20T22:00:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1952025-03-20T22:00:05.321ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1962025-03-20T22:00:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1972025-03-20T22:00:05.322ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1982025-03-20T22:00:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1992025-03-20T22:00:05.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2002025-03-20T22:00:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2012025-03-20T22:00:05.323ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2022025-03-20T22:00:05.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.458602ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
2032025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2042025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2052025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2062025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2072025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2082025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2092025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2102025-03-20T22:00:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2112025-03-20T22:00:05.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2122025-03-20T22:00:05.326ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2132025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2142025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2152025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2162025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2172025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2182025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2192025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2202025-03-20T22:00:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2212025-03-20T22:00:05.330ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2222025-03-20T22:00:05.331ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2232025-03-20T22:00:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2242025-03-20T22:00:05.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2252025-03-20T22:00:05.332ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3549, tv_nsec: 137869724 } })
2262025-03-20T22:00:05.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2272025-03-20T22:00:05.333ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
2282025-03-20T22:00:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2292025-03-20T22:00:05.333ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2302025-03-20T22:00:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2312025-03-20T22:00:05.338ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2322025-03-20T22:00:05.338ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2332025-03-20T22:00:05.338ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2342025-03-20T22:00:05.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
2352025-03-20T22:00:05.340ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2362025-03-20T22:00:05.340ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2372025-03-20T22:00:05.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2382025-03-20T22:00:05.342ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2392025-03-20T22:00:05.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2402025-03-20T22:00:05.344ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2412025-03-20T22:00:05.344ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 104.879856ms iteration = 5 status = {"nsuccess":5}
2422025-03-20T22:00:05.345ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2432025-03-20T22:00:05.345ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2442025-03-20T22:00:05.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2452025-03-20T22:00:05.347ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2462025-03-20T22:00:05.347ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2472025-03-20T22:00:05.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2482025-03-20T22:00:05.351ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2492025-03-20T22:00:05.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2502025-03-20T22:00:05.353ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2512025-03-20T22:00:05.355ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 115.184471ms iteration = 21 status = {"physical_disks_added":0}
2522025-03-20T22:00:05.356ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2532025-03-20T22:00:05.356ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2542025-03-20T22:00:05.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2552025-03-20T22:00:05.358ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2562025-03-20T22:00:05.360ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2572025-03-20T22:00:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2582025-03-20T22:00:05.360ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
2592025-03-20T22:00:05.360ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2602025-03-20T22:00:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
2612025-03-20T22:00:05.367ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2622025-03-20T22:00:05.368ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2632025-03-20T22:00:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2642025-03-20T22:00:05.371ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2652025-03-20T22:00:05.374ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 134.335053ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2662025-03-20T22:00:05.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
2672025-03-20T22:00:05.378ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2682025-03-20T22:00:05.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2692025-03-20T22:00:05.379ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2702025-03-20T22:00:05.379ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2712025-03-20T22:00:05.379ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2722025-03-20T22:00:05.379ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2732025-03-20T22:00:05.380ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 140.296593ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2742025-03-20T22:00:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2752025-03-20T22:00:05.380ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2762025-03-20T22:00:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2772025-03-20T22:00:05.381ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 141.556297ms iteration = 19 status = 0
2782025-03-20T22:00:05.382ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2792025-03-20T22:00:05.383ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 143.069727ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2802025-03-20T22:00:05.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2812025-03-20T22:00:05.384ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2822025-03-20T22:00:05.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2832025-03-20T22:00:05.386ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
2842025-03-20T22:00:05.386ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2852025-03-20T22:00:05.387ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c7c92eb-b32e-4f09-93c3-8205038905ce", "content-length": "26", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
2862025-03-20T22:00:05.387ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2872025-03-20T22:00:05.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2882025-03-20T22:00:05.387ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2892025-03-20T22:00:05.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2902025-03-20T22:00:05.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
2912025-03-20T22:00:05.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2922025-03-20T22:00:05.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2932025-03-20T22:00:05.389ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2942025-03-20T22:00:05.389ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2952025-03-20T22:00:05.390ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2962025-03-20T22:00:05.390ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 150.044593ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2972025-03-20T22:00:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2982025-03-20T22:00:05.392ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2992025-03-20T22:00:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3002025-03-20T22:00:05.393ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012025-03-20T22:00:05.394ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3022025-03-20T22:00:05.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3032025-03-20T22:00:05.394ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3042025-03-20T22:00:05.395ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
3052025-03-20T22:00:05.395ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 154.816661ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3062025-03-20T22:00:05.396ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3072025-03-20T22:00:05.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3082025-03-20T22:00:05.397ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3092025-03-20T22:00:05.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3102025-03-20T22:00:05.399ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3112025-03-20T22:00:05.399ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3122025-03-20T22:00:05.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3132025-03-20T22:00:05.402ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3142025-03-20T22:00:05.402ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3152025-03-20T22:00:05.402ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3162025-03-20T22:00:05.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3172025-03-20T22:00:05.403ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
3182025-03-20T22:00:05.403ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 163.686979ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
3192025-03-20T22:00:05.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3202025-03-20T22:00:05.404ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3212025-03-20T22:00:05.404ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
3222025-03-20T22:00:05.404ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 165.104197ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
3232025-03-20T22:00:05.407ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
3242025-03-20T22:00:05.407ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3252025-03-20T22:00:05.408ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c81da79a-6cae-4808-9af1-b1e2e3aa1459", "content-length": "26", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
3262025-03-20T22:00:05.408ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3272025-03-20T22:00:05.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3282025-03-20T22:00:05.408ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3292025-03-20T22:00:05.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3302025-03-20T22:00:05.410ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3312025-03-20T22:00:05.410ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3322025-03-20T22:00:05.412ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
3332025-03-20T22:00:05.412ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3342025-03-20T22:00:05.412ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-20 21:51:06.492803 UTC dns_group = internal generation = 1
3352025-03-20T22:00:05.412ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 172.619377ms iteration = 10 status = {"generation":1}
3362025-03-20T22:00:05.427ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 187.65332ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
3372025-03-20T22:00:05.435ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
3382025-03-20T22:00:05.435ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 195.136289ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3392025-03-20T22:00:05.437ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3402025-03-20T22:00:05.437ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 0 version = 2
3412025-03-20T22:00:05.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3422025-03-20T22:00:05.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3432025-03-20T22:00:05.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3442025-03-20T22:00:05.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3452025-03-20T22:00:05.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3462025-03-20T22:00:05.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3472025-03-20T22:00:05.457ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-20T21:51:01.993052Z, time_modified: 2025-03-20T21:51:01.993052Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }) }
3482025-03-20T22:00:05.457ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 217.22967ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]}},"default_endpoint":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
3492025-03-20T22:00:05.460ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
3502025-03-20T22:00:05.460ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 220.087877ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3512025-03-20T22:00:05.472ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
3522025-03-20T22:00:05.472ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3532025-03-20T22:00:05.473ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c75fb070-80a4-4baf-ba62-ac37a50c736a", "content-length": "26", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
3542025-03-20T22:00:05.473ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3552025-03-20T22:00:05.476ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
3562025-03-20T22:00:05.563ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3572025-03-20T22:00:05.586ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 346.666972ms iteration = 19 status = {}
3582025-03-20T22:00:05.591ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3592025-03-20T22:00:05.591ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "cebe1b7b-5454-4dd0-82ad-ab6b03de05f0", "content-length": "1", "date": "Thu, 20 Mar 2025 22:00:04 GMT"} })
3602025-03-20T22:00:05.594ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3612025-03-20T22:00:05.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3622025-03-20T22:00:05.615ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5349ea35-c20f-49ae-b3e4-0b49982b0fe3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
3632025-03-20T22:00:05.616ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5349ea35-c20f-49ae-b3e4-0b49982b0fe3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
3642025-03-20T22:00:05.620ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3652025-03-20T22:00:05.621ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "dba780c9-eaa3-4951-87e3-bdb3330a057a", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3662025-03-20T22:00:05.621ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3672025-03-20T22:00:05.623ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3682025-03-20T22:00:05.623ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3692025-03-20T22:00:05.623ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "db382f69-26b9-4b22-ba89-4724c4d212b3", "content-length": "372", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
3702025-03-20T22:00:05.623ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3712025-03-20T22:00:05.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3722025-03-20T22:00:05.624ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "cf3c4502-eb39-47ca-99ea-e213576ac80c", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3732025-03-20T22:00:05.624ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3742025-03-20T22:00:05.627ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
3752025-03-20T22:00:05.627ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 386.814386ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3762025-03-20T22:00:05.669ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3772025-03-20T22:00:05.669ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "67e2486f-d762-431e-84a1-68f13962bc3c", "content-length": "691", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
3782025-03-20T22:00:05.669ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
3792025-03-20T22:00:05.669ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
3802025-03-20T22:00:05.669ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
3812025-03-20T22:00:05.671ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 431.82055ms iteration = 19 status = {}
3822025-03-20T22:00:05.671ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 1 version = 2
3832025-03-20T22:00:05.671ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3842025-03-20T22:00:05.671ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-20 21:51:07.691085 UTC dns_group = external generation = 2
3852025-03-20T22:00:05.671ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 431.647795ms iteration = 10 status = {"generation":2}
3862025-03-20T22:00:05.672ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 432.652472ms iteration = 19 status = null
3872025-03-20T22:00:05.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 58018 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5349ea35-c20f-49ae-b3e4-0b49982b0fe3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
3882025-03-20T22:00:05.720ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3892025-03-20T22:00:05.721ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "6bb58b07-69a8-4add-83e5-6198b7866aa5", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3902025-03-20T22:00:05.721ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3912025-03-20T22:00:05.723ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a9e0fe72-2b53-4df7-a605-9b84430c3fd9", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3922025-03-20T22:00:05.723ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3932025-03-20T22:00:05.725ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "4507f0f7-74de-4fbf-8117-1bf24f117a8e", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3942025-03-20T22:00:05.725ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3952025-03-20T22:00:05.727ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
3962025-03-20T22:00:05.727ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 487.054127ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3972025-03-20T22:00:05.727ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
3982025-03-20T22:00:05.727ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3992025-03-20T22:00:05.727ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a2ebc266-d640-4b86-9381-03891c039494", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
4002025-03-20T22:00:05.727ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4012025-03-20T22:00:05.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4022025-03-20T22:00:05.730ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4032025-03-20T22:00:05.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4042025-03-20T22:00:05.734ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4052025-03-20T22:00:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4062025-03-20T22:00:05.738ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4072025-03-20T22:00:05.738ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4082025-03-20T22:00:05.738ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7c077658-c461-4706-b80d-45ea67aee155", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
4092025-03-20T22:00:05.740ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 500.455825ms iteration = 19 status = {}
4102025-03-20T22:00:05.809ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4112025-03-20T22:00:05.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4122025-03-20T22:00:05.812ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
4132025-03-20T22:00:05.812ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4142025-03-20T22:00:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4152025-03-20T22:00:05.825ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4162025-03-20T22:00:05.826ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "5460d52c-d51d-456e-b476-9976b918532c", "content-length": "134", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
4172025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
4182025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
4192025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
4202025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
4212025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4222025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4232025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4242025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4252025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4262025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4272025-03-20T22:00:05.826ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
4282025-03-20T22:00:05.826ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4292025-03-20T22:00:05.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4302025-03-20T22:00:05.829ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4312025-03-20T22:00:05.830ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0474f5f3-651d-4b25-b322-e1bd43939a3b", "content-length": "2", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
4322025-03-20T22:00:05.830ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4332025-03-20T22:00:05.830ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5304b1f-c346-40c2-b555-fed2bc56e3c4", "content-length": "52", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
4342025-03-20T22:00:05.830ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
4352025-03-20T22:00:05.830ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4362025-03-20T22:00:06.041ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "885435a9-b578-492f-a92f-2af491e4c1fb", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
4372025-03-20T22:00:06.041ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
4382025-03-20T22:00:06.041ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4392025-03-20T22:00:06.041ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f1d9fca-0771-466b-bbb2-3d5fb68265dd", "content-length": "682", "date": "Thu, 20 Mar 2025 22:00:05 GMT"} })
4402025-03-20T22:00:06.041ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4412025-03-20T22:00:06.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4422025-03-20T22:00:06.047ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4432025-03-20T22:00:06.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4442025-03-20T22:00:06.052ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4452025-03-20T22:00:06.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4462025-03-20T22:00:06.055ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4472025-03-20T22:00:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4482025-03-20T22:00:06.066ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 826.214789ms iteration = 19 status = {}
4492025-03-20T22:00:06.698ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 73c0c3a4-6ed3-4960-b399-586d1ac0f294 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4502025-03-20T22:00:06.698ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 73c0c3a4-6ed3-4960-b399-586d1ac0f294 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4512025-03-20T22:00:06.701ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2635 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 73c0c3a4-6ed3-4960-b399-586d1ac0f294 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4522025-03-20T22:00:07.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 567 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 55f9406d-088d-498c-8610-be993ecaa1c4 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
4532025-03-20T22:00:07.720ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a99b88af-de30-48ba-bf91-0eea29fac656 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4542025-03-20T22:00:07.720ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a99b88af-de30-48ba-bf91-0eea29fac656 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4552025-03-20T22:00:07.722ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a99b88af-de30-48ba-bf91-0eea29fac656 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4562025-03-20T22:00:08.743ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b57cdac1-7591-4106-ac9f-71f595dcf351 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4572025-03-20T22:00:08.743ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b57cdac1-7591-4106-ac9f-71f595dcf351 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4582025-03-20T22:00:08.745ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2669 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b57cdac1-7591-4106-ac9f-71f595dcf351 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4592025-03-20T22:00:09.765ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0ba6b807-4bb6-4f55-b1cd-a0ed1f0499fe roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4602025-03-20T22:00:09.765ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0ba6b807-4bb6-4f55-b1cd-a0ed1f0499fe resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4612025-03-20T22:00:09.767ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2634 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0ba6b807-4bb6-4f55-b1cd-a0ed1f0499fe response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4622025-03-20T22:00:10.787ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = dadc1535-47bb-47d5-a79e-f0de44ac0b63 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4632025-03-20T22:00:10.787ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = dadc1535-47bb-47d5-a79e-f0de44ac0b63 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4642025-03-20T22:00:10.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2541 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = dadc1535-47bb-47d5-a79e-f0de44ac0b63 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4652025-03-20T22:00:11.808ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2f2f68c9-29f8-45d3-8076-3ce9789bd8cb roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4662025-03-20T22:00:11.809ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2f2f68c9-29f8-45d3-8076-3ce9789bd8cb resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4672025-03-20T22:00:11.811ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2468 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2f2f68c9-29f8-45d3-8076-3ce9789bd8cb response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4682025-03-20T22:00:12.830ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6327167e-777f-4ec6-ad0d-aa56d83b805c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4692025-03-20T22:00:12.831ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6327167e-777f-4ec6-ad0d-aa56d83b805c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4702025-03-20T22:00:12.833ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2342 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6327167e-777f-4ec6-ad0d-aa56d83b805c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4712025-03-20T22:00:13.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a291ebbb-9316-4e03-90fc-676a93a34c24 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4722025-03-20T22:00:13.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a291ebbb-9316-4e03-90fc-676a93a34c24 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4732025-03-20T22:00:13.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2283 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a291ebbb-9316-4e03-90fc-676a93a34c24 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4742025-03-20T22:00:14.873ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8e2606a-3061-468a-9330-f4d608e9bdf7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4752025-03-20T22:00:14.874ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8e2606a-3061-468a-9330-f4d608e9bdf7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4762025-03-20T22:00:14.875ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1660 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8e2606a-3061-468a-9330-f4d608e9bdf7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4772025-03-20T22:00:15.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
4782025-03-20T22:00:15.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4792025-03-20T22:00:15.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
4802025-03-20T22:00:15.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4812025-03-20T22:00:15.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4822025-03-20T22:00:15.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4832025-03-20T22:00:15.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4842025-03-20T22:00:15.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4852025-03-20T22:00:15.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4862025-03-20T22:00:15.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4872025-03-20T22:00:15.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4882025-03-20T22:00:15.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4892025-03-20T22:00:15.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4902025-03-20T22:00:15.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
4912025-03-20T22:00:15.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4922025-03-20T22:00:15.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4932025-03-20T22:00:15.264ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
4942025-03-20T22:00:15.264ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.626939ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
4952025-03-20T22:00:15.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4962025-03-20T22:00:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4972025-03-20T22:00:15.309ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.927121ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4982025-03-20T22:00:15.895ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c932959a-eba4-4b1f-b00b-1801c5046718 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
4992025-03-20T22:00:15.895ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c932959a-eba4-4b1f-b00b-1801c5046718 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5002025-03-20T22:00:15.897ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1840 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c932959a-eba4-4b1f-b00b-1801c5046718 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5012025-03-20T22:00:16.918ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 88fe3558-330e-4051-86bc-d209d6b28e81 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5022025-03-20T22:00:16.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 88fe3558-330e-4051-86bc-d209d6b28e81 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5032025-03-20T22:00:16.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2148 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 88fe3558-330e-4051-86bc-d209d6b28e81 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5042025-03-20T22:00:17.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 474 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 19395e07-4bc0-467d-899b-0ba154a9cdd5 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
5052025-03-20T22:00:17.940ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a0d912ba-b094-47c3-9628-6b5f6ef64d00 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5062025-03-20T22:00:17.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a0d912ba-b094-47c3-9628-6b5f6ef64d00 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5072025-03-20T22:00:17.943ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2157 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a0d912ba-b094-47c3-9628-6b5f6ef64d00 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5082025-03-20T22:00:18.963ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 596de604-c95a-4d7b-a526-451e1604cfbe roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5092025-03-20T22:00:18.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 596de604-c95a-4d7b-a526-451e1604cfbe resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5102025-03-20T22:00:18.964ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1674 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 596de604-c95a-4d7b-a526-451e1604cfbe response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5112025-03-20T22:00:19.983ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9c19c5ba-be0d-4b0d-9ec6-73e219d79b6c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5122025-03-20T22:00:19.983ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9c19c5ba-be0d-4b0d-9ec6-73e219d79b6c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5132025-03-20T22:00:19.985ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2479 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9c19c5ba-be0d-4b0d-9ec6-73e219d79b6c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5142025-03-20T22:00:21.005ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9b6ad590-4e56-438e-aef4-7059c66b7b60 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5152025-03-20T22:00:21.006ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9b6ad590-4e56-438e-aef4-7059c66b7b60 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5162025-03-20T22:00:21.007ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2001 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9b6ad590-4e56-438e-aef4-7059c66b7b60 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5172025-03-20T22:00:22.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a0a577c3-81a6-4c14-a37c-022c99c330bf roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5182025-03-20T22:00:22.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a0a577c3-81a6-4c14-a37c-022c99c330bf resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5192025-03-20T22:00:22.029ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a0a577c3-81a6-4c14-a37c-022c99c330bf response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5202025-03-20T22:00:23.051ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 549267a1-4a83-4bb5-9c4c-715609d28d46 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5212025-03-20T22:00:23.051ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 549267a1-4a83-4bb5-9c4c-715609d28d46 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5222025-03-20T22:00:23.052ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1608 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 549267a1-4a83-4bb5-9c4c-715609d28d46 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5232025-03-20T22:00:24.072ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 29b9a66c-d5f8-4608-839b-a34ca318f77a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5242025-03-20T22:00:24.072ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 29b9a66c-d5f8-4608-839b-a34ca318f77a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5252025-03-20T22:00:24.074ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2112 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 29b9a66c-d5f8-4608-839b-a34ca318f77a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5262025-03-20T22:00:25.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 11fbfaf1-59f5-4fab-b067-1ae39ad327a5 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5272025-03-20T22:00:25.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 11fbfaf1-59f5-4fab-b067-1ae39ad327a5 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5282025-03-20T22:00:25.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2302 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 11fbfaf1-59f5-4fab-b067-1ae39ad327a5 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5292025-03-20T22:00:25.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
5302025-03-20T22:00:25.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
5312025-03-20T22:00:25.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5322025-03-20T22:00:25.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5332025-03-20T22:00:25.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5342025-03-20T22:00:25.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5352025-03-20T22:00:25.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5362025-03-20T22:00:25.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5372025-03-20T22:00:25.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5382025-03-20T22:00:25.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5392025-03-20T22:00:25.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5402025-03-20T22:00:25.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5412025-03-20T22:00:25.269ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5422025-03-20T22:00:25.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5432025-03-20T22:00:25.271ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.544641ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5442025-03-20T22:00:25.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5452025-03-20T22:00:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
5462025-03-20T22:00:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5472025-03-20T22:00:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5482025-03-20T22:00:25.296ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
5492025-03-20T22:00:25.296ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.719523ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
5502025-03-20T22:00:26.116ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee635036-73aa-4b6b-b8c3-83ad81aff41c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5512025-03-20T22:00:26.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee635036-73aa-4b6b-b8c3-83ad81aff41c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5522025-03-20T22:00:26.118ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2115 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee635036-73aa-4b6b-b8c3-83ad81aff41c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5532025-03-20T22:00:27.138ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cd8e19c7-5464-4328-b115-56057ffa81f3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5542025-03-20T22:00:27.138ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cd8e19c7-5464-4328-b115-56057ffa81f3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5552025-03-20T22:00:27.139ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1764 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cd8e19c7-5464-4328-b115-56057ffa81f3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5562025-03-20T22:00:27.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 538 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 7ed447d0-d92a-4aa6-82e0-9e4ae6316f67 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
5572025-03-20T22:00:28.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b9ebaec6-1a8f-4eb5-a5e5-ff813ae86bcb roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5582025-03-20T22:00:28.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b9ebaec6-1a8f-4eb5-a5e5-ff813ae86bcb resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5592025-03-20T22:00:28.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1995 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b9ebaec6-1a8f-4eb5-a5e5-ff813ae86bcb response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5602025-03-20T22:00:29.180ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e5b0d999-1884-4b47-a0e8-f05052c2533a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5612025-03-20T22:00:29.180ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e5b0d999-1884-4b47-a0e8-f05052c2533a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5622025-03-20T22:00:29.182ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2449 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e5b0d999-1884-4b47-a0e8-f05052c2533a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5632025-03-20T22:00:30.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 47381b7b-b84a-4699-b204-7dbb50fa14f3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5642025-03-20T22:00:30.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 47381b7b-b84a-4699-b204-7dbb50fa14f3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5652025-03-20T22:00:30.206ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2619 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 47381b7b-b84a-4699-b204-7dbb50fa14f3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5662025-03-20T22:00:31.224ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1673c165-b05b-4908-8832-f665c2c6aac6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5672025-03-20T22:00:31.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1673c165-b05b-4908-8832-f665c2c6aac6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5682025-03-20T22:00:31.226ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2087 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1673c165-b05b-4908-8832-f665c2c6aac6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5692025-03-20T22:00:32.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ab1a9a53-ba9e-41bf-af83-993f019c0630 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5702025-03-20T22:00:32.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ab1a9a53-ba9e-41bf-af83-993f019c0630 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5712025-03-20T22:00:32.248ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:54401
5722025-03-20T22:00:32.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54401 req_id = e08da64c-e6c2-4eb2-ae2f-b9b4153ad3e4 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
5732025-03-20T22:00:32.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54401 req_id = e08da64c-e6c2-4eb2-ae2f-b9b4153ad3e4 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
5742025-03-20T22:00:32.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2141 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ab1a9a53-ba9e-41bf-af83-993f019c0630 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5752025-03-20T22:00:32.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2807 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54401 req_id = e08da64c-e6c2-4eb2-ae2f-b9b4153ad3e4 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
5762025-03-20T22:00:32.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54401 req_id = 4a1e6619-6a0f-43a7-9d0d-48e4f0749339 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
5772025-03-20T22:00:32.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54401 req_id = 4a1e6619-6a0f-43a7-9d0d-48e4f0749339 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
5782025-03-20T22:00:32.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1876 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:54401 req_id = 4a1e6619-6a0f-43a7-9d0d-48e4f0749339 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
5792025-03-20T22:00:33.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5657072b-9b02-4599-b986-398460fd1149 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5802025-03-20T22:00:33.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5657072b-9b02-4599-b986-398460fd1149 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5812025-03-20T22:00:33.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1945 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5657072b-9b02-4599-b986-398460fd1149 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5822025-03-20T22:00:34.290ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fe3feace-0893-41ff-9cd4-bdf5f8c04d2a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5832025-03-20T22:00:34.291ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fe3feace-0893-41ff-9cd4-bdf5f8c04d2a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5842025-03-20T22:00:34.292ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1998 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fe3feace-0893-41ff-9cd4-bdf5f8c04d2a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
5852025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
5862025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5872025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5882025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
5892025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
5902025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5912025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5922025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
5932025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5942025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
5952025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
5962025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5972025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
5982025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5992025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6002025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
6012025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
6022025-03-20T22:00:35.238ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6032025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
6042025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6052025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
6062025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
6072025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
6082025-03-20T22:00:35.239ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
6092025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6102025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6112025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6122025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
6132025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6142025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
6152025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6162025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
6172025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6182025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
6192025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
6202025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
6212025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
6222025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
6232025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
6242025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6252025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6262025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6272025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6282025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6292025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6302025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6312025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6322025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6332025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6342025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6352025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
6362025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6372025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6382025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
6392025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
6402025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6412025-03-20T22:00:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6422025-03-20T22:00:35.241ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6432025-03-20T22:00:35.242ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6442025-03-20T22:00:35.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6452025-03-20T22:00:35.243ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.193052ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
6462025-03-20T22:00:35.244ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6472025-03-20T22:00:35.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6482025-03-20T22:00:35.245ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6492025-03-20T22:00:35.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6502025-03-20T22:00:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6512025-03-20T22:00:35.246ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6522025-03-20T22:00:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6532025-03-20T22:00:35.255ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3579, tv_nsec: 61198603 } })
6542025-03-20T22:00:35.256ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6552025-03-20T22:00:35.256ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
6562025-03-20T22:00:35.259ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6572025-03-20T22:00:35.259ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6582025-03-20T22:00:35.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6592025-03-20T22:00:35.263ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6602025-03-20T22:00:35.263ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3579, tv_nsec: 68640652 } })
6612025-03-20T22:00:35.263ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
6622025-03-20T22:00:35.265ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6632025-03-20T22:00:35.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6642025-03-20T22:00:35.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6652025-03-20T22:00:35.266ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6662025-03-20T22:00:35.266ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6672025-03-20T22:00:35.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6682025-03-20T22:00:35.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6692025-03-20T22:00:35.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6702025-03-20T22:00:35.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6712025-03-20T22:00:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6722025-03-20T22:00:35.303ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6732025-03-20T22:00:35.304ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6742025-03-20T22:00:35.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6752025-03-20T22:00:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
6762025-03-20T22:00:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6772025-03-20T22:00:35.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6782025-03-20T22:00:35.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6792025-03-20T22:00:35.310ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6802025-03-20T22:00:35.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6812025-03-20T22:00:35.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6822025-03-20T22:00:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6832025-03-20T22:00:35.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c84f9bc-4db6-48e9-af3f-8b49a75132e7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
6842025-03-20T22:00:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6852025-03-20T22:00:35.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c84f9bc-4db6-48e9-af3f-8b49a75132e7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
6862025-03-20T22:00:35.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6872025-03-20T22:00:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6882025-03-20T22:00:35.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 75.345595ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
6892025-03-20T22:00:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6902025-03-20T22:00:35.314ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6912025-03-20T22:00:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6922025-03-20T22:00:35.315ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 76.237929ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
6932025-03-20T22:00:35.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6942025-03-20T22:00:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6952025-03-20T22:00:35.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4317 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c84f9bc-4db6-48e9-af3f-8b49a75132e7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
6962025-03-20T22:00:35.318ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6972025-03-20T22:00:35.318ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6982025-03-20T22:00:35.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6992025-03-20T22:00:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7002025-03-20T22:00:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7012025-03-20T22:00:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7022025-03-20T22:00:35.319ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3579, tv_nsec: 124772564 } })
7032025-03-20T22:00:35.319ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
7042025-03-20T22:00:35.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7052025-03-20T22:00:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7062025-03-20T22:00:35.322ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7072025-03-20T22:00:35.322ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
7082025-03-20T22:00:35.322ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7092025-03-20T22:00:35.322ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7102025-03-20T22:00:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7112025-03-20T22:00:35.323ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fb5689c-9ef6-4469-95c9-264825e8606b", "content-length": "26", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
7122025-03-20T22:00:35.323ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7132025-03-20T22:00:35.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 85.731483ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7142025-03-20T22:00:35.325ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7152025-03-20T22:00:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
7162025-03-20T22:00:35.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7172025-03-20T22:00:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7182025-03-20T22:00:35.325ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7192025-03-20T22:00:35.325ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 86.593296ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7202025-03-20T22:00:35.326ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 88.115487ms iteration = 20 status = 0
7212025-03-20T22:00:35.327ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 88.825996ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
7222025-03-20T22:00:35.330ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7232025-03-20T22:00:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7242025-03-20T22:00:35.337ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7252025-03-20T22:00:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7262025-03-20T22:00:35.339ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7272025-03-20T22:00:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7282025-03-20T22:00:35.341ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7292025-03-20T22:00:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7302025-03-20T22:00:35.343ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7312025-03-20T22:00:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7322025-03-20T22:00:35.353ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 114.814211ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7332025-03-20T22:00:35.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 120.486463ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7342025-03-20T22:00:35.366ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
7352025-03-20T22:00:35.366ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7362025-03-20T22:00:35.366ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6c16f91-1eb6-4c71-a821-5a7802f3f6c8", "content-length": "26", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
7372025-03-20T22:00:35.366ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7382025-03-20T22:00:35.368ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7392025-03-20T22:00:35.371ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7402025-03-20T22:00:35.371ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7412025-03-20T22:00:35.371ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7422025-03-20T22:00:35.371ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7432025-03-20T22:00:35.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7442025-03-20T22:00:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7452025-03-20T22:00:35.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462025-03-20T22:00:35.376ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
7472025-03-20T22:00:35.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7482025-03-20T22:00:35.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7492025-03-20T22:00:35.385ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
7502025-03-20T22:00:35.385ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 146.252362ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7512025-03-20T22:00:35.387ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
7522025-03-20T22:00:35.387ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 148.012869ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
7532025-03-20T22:00:35.402ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 163.461672ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
7542025-03-20T22:00:35.405ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
7552025-03-20T22:00:35.405ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7562025-03-20T22:00:35.405ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "361e7fb3-bc63-4c4d-bc80-9c1c9594bd49", "content-length": "26", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
7572025-03-20T22:00:35.405ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7582025-03-20T22:00:35.407ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7592025-03-20T22:00:35.407ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7602025-03-20T22:00:35.407ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 168.367233ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7612025-03-20T22:00:35.407ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
7622025-03-20T22:00:35.416ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7632025-03-20T22:00:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7642025-03-20T22:00:35.428ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7652025-03-20T22:00:35.428ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6f671f2-f5f0-498f-95c8-4370db344468", "content-length": "1", "date": "Thu, 20 Mar 2025 22:00:34 GMT"} })
7662025-03-20T22:00:35.430ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7672025-03-20T22:00:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7682025-03-20T22:00:35.432ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 193.66414ms iteration = 20 status = null
7692025-03-20T22:00:35.439ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 200.679748ms iteration = 22 status = {"physical_disks_added":0}
7702025-03-20T22:00:35.450ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7712025-03-20T22:00:35.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7722025-03-20T22:00:35.454ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7732025-03-20T22:00:35.456ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7742025-03-20T22:00:35.456ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7752025-03-20T22:00:35.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7762025-03-20T22:00:35.467ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 228.79953ms iteration = 20 status = {}
7772025-03-20T22:00:35.558ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7782025-03-20T22:00:35.558ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7792025-03-20T22:00:35.558ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b7ff8c6-663e-4b9f-9e43-085e301128fc", "content-length": "372", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
7802025-03-20T22:00:35.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7812025-03-20T22:00:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7822025-03-20T22:00:35.559ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7832025-03-20T22:00:35.562ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7842025-03-20T22:00:35.562ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7852025-03-20T22:00:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7862025-03-20T22:00:35.610ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
7872025-03-20T22:00:35.610ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7882025-03-20T22:00:35.610ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "740d5799-4d0d-456b-884c-c51607452506", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
7892025-03-20T22:00:35.610ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7902025-03-20T22:00:35.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7912025-03-20T22:00:35.613ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7922025-03-20T22:00:35.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7932025-03-20T22:00:35.616ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7942025-03-20T22:00:35.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7952025-03-20T22:00:35.620ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7962025-03-20T22:00:35.620ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7972025-03-20T22:00:35.620ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "31ab3cfb-1510-436f-8f45-8b9c9b58f969", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
7982025-03-20T22:00:35.622ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 383.290261ms iteration = 20 status = {}
7992025-03-20T22:00:35.639ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8002025-03-20T22:00:35.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8012025-03-20T22:00:35.641ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
8022025-03-20T22:00:35.641ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8032025-03-20T22:00:35.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8042025-03-20T22:00:35.651ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8052025-03-20T22:00:35.651ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f4b3dfc-7fff-4a42-b21a-27ce6bd9f921", "content-length": "134", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
8062025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
8072025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
8082025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8092025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8102025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8112025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8122025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8132025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8142025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8152025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8162025-03-20T22:00:35.652ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
8172025-03-20T22:00:35.652ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8182025-03-20T22:00:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8192025-03-20T22:00:35.654ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8202025-03-20T22:00:35.655ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "35f2ac2f-a825-4c8b-8a50-ca58fe2ca083", "content-length": "2", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
8212025-03-20T22:00:35.655ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8222025-03-20T22:00:35.655ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b2b8488-d9ba-47a1-aab3-f9a5b66300b1", "content-length": "52", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
8232025-03-20T22:00:35.655ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
8242025-03-20T22:00:35.655ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8252025-03-20T22:00:35.673ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8262025-03-20T22:00:35.752ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "835ad420-ddad-48a5-a921-3de572988d97", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
8272025-03-20T22:00:35.752ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
8282025-03-20T22:00:35.752ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8292025-03-20T22:00:35.752ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c176002-bf04-4358-837c-96833febf61e", "content-length": "682", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
8302025-03-20T22:00:35.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8312025-03-20T22:00:35.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8322025-03-20T22:00:35.753ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5511a300-a9a2-4e9c-80a2-0ca4f46a6878", "content-length": "691", "date": "Thu, 20 Mar 2025 22:00:35 GMT"} })
8332025-03-20T22:00:35.753ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8342025-03-20T22:00:35.753ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8352025-03-20T22:00:35.753ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8362025-03-20T22:00:35.755ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 516.289509ms iteration = 20 status = {}
8372025-03-20T22:00:35.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8382025-03-20T22:00:35.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8392025-03-20T22:00:35.760ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8402025-03-20T22:00:35.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8412025-03-20T22:00:35.761ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8422025-03-20T22:00:35.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8432025-03-20T22:00:35.768ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 529.747839ms iteration = 20 status = {}
8442025-03-20T22:00:36.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 72da557d-2564-4343-9a34-08cb204b7e5a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8452025-03-20T22:00:36.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 72da557d-2564-4343-9a34-08cb204b7e5a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8462025-03-20T22:00:36.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1758 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 72da557d-2564-4343-9a34-08cb204b7e5a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8472025-03-20T22:00:37.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 462 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 67846b50-87b6-48db-b9e5-6dca303c5216 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
8482025-03-20T22:00:37.365ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a46b7ae-fb0e-4aeb-b062-f46204d0f481 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8492025-03-20T22:00:37.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a46b7ae-fb0e-4aeb-b062-f46204d0f481 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8502025-03-20T22:00:37.367ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2008 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a46b7ae-fb0e-4aeb-b062-f46204d0f481 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8512025-03-20T22:00:38.387ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e6e356e3-2cc4-44f3-87e0-73c8707974a1 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8522025-03-20T22:00:38.388ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e6e356e3-2cc4-44f3-87e0-73c8707974a1 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8532025-03-20T22:00:38.390ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2169 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e6e356e3-2cc4-44f3-87e0-73c8707974a1 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8542025-03-20T22:00:39.409ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0e31133-02ce-4411-8c5b-9738e1048b7f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8552025-03-20T22:00:39.410ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0e31133-02ce-4411-8c5b-9738e1048b7f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8562025-03-20T22:00:39.411ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0e31133-02ce-4411-8c5b-9738e1048b7f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8572025-03-20T22:00:40.432ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c31372a8-58f2-40dd-b438-e255d288f5ee roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8582025-03-20T22:00:40.432ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c31372a8-58f2-40dd-b438-e255d288f5ee resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8592025-03-20T22:00:40.434ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1934 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c31372a8-58f2-40dd-b438-e255d288f5ee response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8602025-03-20T22:00:41.458ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3cd732e8-17b9-412d-a6dd-63a259c30654 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8612025-03-20T22:00:41.459ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3cd732e8-17b9-412d-a6dd-63a259c30654 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8622025-03-20T22:00:41.461ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2458 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3cd732e8-17b9-412d-a6dd-63a259c30654 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8632025-03-20T22:00:42.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f476f9f0-95a5-4efb-ab94-45d576f62277 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8642025-03-20T22:00:42.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f476f9f0-95a5-4efb-ab94-45d576f62277 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8652025-03-20T22:00:42.483ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1980 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f476f9f0-95a5-4efb-ab94-45d576f62277 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8662025-03-20T22:00:43.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4b682d19-e404-4f3c-9e17-bbfbb6b033e0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8672025-03-20T22:00:43.503ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4b682d19-e404-4f3c-9e17-bbfbb6b033e0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8682025-03-20T22:00:43.505ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2037 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4b682d19-e404-4f3c-9e17-bbfbb6b033e0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8692025-03-20T22:00:44.525ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 10bcfa2d-9945-44a7-abe8-da579396a444 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8702025-03-20T22:00:44.525ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 10bcfa2d-9945-44a7-abe8-da579396a444 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8712025-03-20T22:00:44.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1738 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 10bcfa2d-9945-44a7-abe8-da579396a444 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8722025-03-20T22:00:45.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
8732025-03-20T22:00:45.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8742025-03-20T22:00:45.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
8752025-03-20T22:00:45.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8762025-03-20T22:00:45.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8772025-03-20T22:00:45.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8782025-03-20T22:00:45.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8792025-03-20T22:00:45.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8802025-03-20T22:00:45.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8812025-03-20T22:00:45.242ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.316428ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8822025-03-20T22:00:45.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8832025-03-20T22:00:45.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8842025-03-20T22:00:45.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8852025-03-20T22:00:45.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8862025-03-20T22:00:45.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8872025-03-20T22:00:45.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8882025-03-20T22:00:45.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
8892025-03-20T22:00:45.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8902025-03-20T22:00:45.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8912025-03-20T22:00:45.257ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
8922025-03-20T22:00:45.257ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.526436ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
8932025-03-20T22:00:45.547ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b3a64188-158a-4382-8fec-05f64648ee96 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8942025-03-20T22:00:45.547ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b3a64188-158a-4382-8fec-05f64648ee96 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8952025-03-20T22:00:45.549ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2186 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b3a64188-158a-4382-8fec-05f64648ee96 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8962025-03-20T22:00:46.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8ea5ab0b-e89d-4475-9ba6-eff2603b771a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8972025-03-20T22:00:46.569ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8ea5ab0b-e89d-4475-9ba6-eff2603b771a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8982025-03-20T22:00:46.570ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2054 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8ea5ab0b-e89d-4475-9ba6-eff2603b771a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
8992025-03-20T22:00:47.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 478 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 2aa38a53-4564-4fd8-a7ff-56475775b0cf response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
9002025-03-20T22:00:47.590ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 39c94d7d-22ac-4e1a-913c-f9206d2c582a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9012025-03-20T22:00:47.590ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 39c94d7d-22ac-4e1a-913c-f9206d2c582a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9022025-03-20T22:00:47.592ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1887 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 39c94d7d-22ac-4e1a-913c-f9206d2c582a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9032025-03-20T22:00:48.612ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d209dc0-8a0d-4e50-bb7d-56b44a78e57e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9042025-03-20T22:00:48.612ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d209dc0-8a0d-4e50-bb7d-56b44a78e57e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9052025-03-20T22:00:48.614ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2082 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d209dc0-8a0d-4e50-bb7d-56b44a78e57e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9062025-03-20T22:00:49.633ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 474df5b6-9485-45dc-ab7c-15ee5bfbbe65 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9072025-03-20T22:00:49.633ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 474df5b6-9485-45dc-ab7c-15ee5bfbbe65 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9082025-03-20T22:00:49.635ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2177 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 474df5b6-9485-45dc-ab7c-15ee5bfbbe65 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9092025-03-20T22:00:50.653ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e71b2267-7bba-461a-bd99-d99f2b7bc61e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9102025-03-20T22:00:50.654ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e71b2267-7bba-461a-bd99-d99f2b7bc61e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9112025-03-20T22:00:50.656ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2335 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e71b2267-7bba-461a-bd99-d99f2b7bc61e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9122025-03-20T22:00:51.675ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 60dded2a-80b6-488d-8a05-bf4e9a018a2e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9132025-03-20T22:00:51.675ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 60dded2a-80b6-488d-8a05-bf4e9a018a2e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9142025-03-20T22:00:51.677ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2009 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 60dded2a-80b6-488d-8a05-bf4e9a018a2e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9152025-03-20T22:00:52.697ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22975831-d23f-4476-9203-ad3bed1144e3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9162025-03-20T22:00:52.698ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22975831-d23f-4476-9203-ad3bed1144e3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9172025-03-20T22:00:52.699ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1956 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22975831-d23f-4476-9203-ad3bed1144e3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9182025-03-20T22:00:53.718ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bda4a10c-5033-451d-8fc3-f9c135680c07 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9192025-03-20T22:00:53.719ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bda4a10c-5033-451d-8fc3-f9c135680c07 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9202025-03-20T22:00:53.721ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2185 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bda4a10c-5033-451d-8fc3-f9c135680c07 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9212025-03-20T22:00:54.740ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c58d92e7-1cd8-40e2-aa80-0b1f63c03501 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9222025-03-20T22:00:54.740ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c58d92e7-1cd8-40e2-aa80-0b1f63c03501 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9232025-03-20T22:00:54.742ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2020 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c58d92e7-1cd8-40e2-aa80-0b1f63c03501 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9242025-03-20T22:00:55.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
9252025-03-20T22:00:55.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9262025-03-20T22:00:55.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
9272025-03-20T22:00:55.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9282025-03-20T22:00:55.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9292025-03-20T22:00:55.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9302025-03-20T22:00:55.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9312025-03-20T22:00:55.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322025-03-20T22:00:55.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9332025-03-20T22:00:55.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9342025-03-20T22:00:55.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9352025-03-20T22:00:55.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9362025-03-20T22:00:55.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9372025-03-20T22:00:55.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9382025-03-20T22:00:55.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9392025-03-20T22:00:55.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
9402025-03-20T22:00:55.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9412025-03-20T22:00:55.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9422025-03-20T22:00:55.256ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
9432025-03-20T22:00:55.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.407035ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
9442025-03-20T22:00:55.271ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.545551ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9452025-03-20T22:00:55.761ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4bfd6cd1-f803-4583-90ab-ff8a3e6dbb44 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9462025-03-20T22:00:55.762ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4bfd6cd1-f803-4583-90ab-ff8a3e6dbb44 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9472025-03-20T22:00:55.763ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2059 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4bfd6cd1-f803-4583-90ab-ff8a3e6dbb44 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9482025-03-20T22:00:56.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8bebb084-1548-48d7-a018-34a1aed33031 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9492025-03-20T22:00:56.782ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8bebb084-1548-48d7-a018-34a1aed33031 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9502025-03-20T22:00:56.784ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1986 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8bebb084-1548-48d7-a018-34a1aed33031 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9512025-03-20T22:00:57.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 562 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 61fd2a8b-4ea3-42ac-b4a5-d588462f5cec response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
9522025-03-20T22:00:57.803ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1aec0c4f-f799-47c4-8e9d-23037d207809 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9532025-03-20T22:00:57.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1aec0c4f-f799-47c4-8e9d-23037d207809 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9542025-03-20T22:00:57.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2042 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1aec0c4f-f799-47c4-8e9d-23037d207809 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9552025-03-20T22:00:58.825ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 778988c5-1ec5-4cb8-88b2-a19431d916cd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9562025-03-20T22:00:58.826ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 778988c5-1ec5-4cb8-88b2-a19431d916cd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9572025-03-20T22:00:58.828ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2675 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 778988c5-1ec5-4cb8-88b2-a19431d916cd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9582025-03-20T22:00:59.847ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d9e1c52a-04bc-469d-ac45-754268bb679f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9592025-03-20T22:00:59.848ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d9e1c52a-04bc-469d-ac45-754268bb679f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9602025-03-20T22:00:59.850ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2653 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d9e1c52a-04bc-469d-ac45-754268bb679f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9612025-03-20T22:01:00.870ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1a79a651-4f8f-4791-8575-d65b05868fa5 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9622025-03-20T22:01:00.870ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1a79a651-4f8f-4791-8575-d65b05868fa5 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9632025-03-20T22:01:00.872ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2180 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1a79a651-4f8f-4791-8575-d65b05868fa5 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9642025-03-20T22:01:01.891ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5cfcb0a4-4dca-40ee-81b7-a2f32462b9a7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9652025-03-20T22:01:01.891ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5cfcb0a4-4dca-40ee-81b7-a2f32462b9a7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9662025-03-20T22:01:01.893ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1766 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5cfcb0a4-4dca-40ee-81b7-a2f32462b9a7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9672025-03-20T22:01:02.913ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8328c9d8-1167-43e9-873b-623255d764fc roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9682025-03-20T22:01:02.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8328c9d8-1167-43e9-873b-623255d764fc resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9692025-03-20T22:01:02.915ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1742 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8328c9d8-1167-43e9-873b-623255d764fc response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9702025-03-20T22:01:03.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b92e9b90-a31f-4573-ac57-87550e9adf97 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9712025-03-20T22:01:03.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b92e9b90-a31f-4573-ac57-87550e9adf97 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9722025-03-20T22:01:03.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2050 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b92e9b90-a31f-4573-ac57-87550e9adf97 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9732025-03-20T22:01:04.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0cc86fe-f2d9-4114-8cf4-3f43d50e010b roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9742025-03-20T22:01:04.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0cc86fe-f2d9-4114-8cf4-3f43d50e010b resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9752025-03-20T22:01:04.959ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2187 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0cc86fe-f2d9-4114-8cf4-3f43d50e010b response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
9762025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
9772025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9782025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
9792025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9802025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
9812025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
9822025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9832025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
9842025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9852025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
9862025-03-20T22:01:05.238ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-20 21:51:05.238845974 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9872025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
9882025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9892025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
9902025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
9912025-03-20T22:01:05.238ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
9922025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
9932025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
9942025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9952025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
9962025-03-20T22:01:05.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9972025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
9982025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
9992025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10002025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10012025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
10022025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10032025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10042025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
10052025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10062025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Timeout
10072025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
10082025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
10092025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10102025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
10112025-03-20T22:01:05.239ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f
10122025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.31\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f"}
10132025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
10142025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
10152025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
10162025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10172025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
10182025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10192025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
10202025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10212025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
10222025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10232025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
10242025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10252025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
10262025-03-20T22:01:05.239ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
10272025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10282025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
10292025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10302025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
10312025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10322025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10332025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
10342025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
10352025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10362025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10372025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10382025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10392025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
10402025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
10412025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
10422025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10432025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
10442025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10452025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
10462025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10472025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
10482025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
10492025-03-20T22:01:05.239ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
10502025-03-20T22:01:05.239ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
10512025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
10522025-03-20T22:01:05.239ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
10532025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
10542025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10552025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
10562025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10572025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10582025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
10592025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10602025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10612025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10622025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10632025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10642025-03-20T22:01:05.239ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55400d62-c4af-483b-8223-1234d14cf0ad.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3609, tv_nsec: 45196685 } })
10652025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10662025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
10672025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
10682025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10692025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
10702025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10712025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10722025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10732025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10742025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10752025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10762025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10772025-03-20T22:01:05.239ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
10782025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10792025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10802025-03-20T22:01:05.239ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58723511-3cb5-4c26-bebf-16620d9cd641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("59acd018-3b3d-4cb6-bd0f-2dfaf2b8d965.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 3609, tv_nsec: 45371140 } })
10812025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
10822025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10832025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10842025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10852025-03-20T22:01:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10862025-03-20T22:01:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10872025-03-20T22:01:05.240ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
10882025-03-20T22:01:05.240ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 862.153\u{b5}s iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
10892025-03-20T22:01:05.240ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
10902025-03-20T22:01:05.240ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.09526ms iteration = 11 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
10912025-03-20T22:01:05.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10922025-03-20T22:01:05.242ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.868773ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
10932025-03-20T22:01:05.243ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10942025-03-20T22:01:05.243ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.778038ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10952025-03-20T22:01:05.243ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10962025-03-20T22:01:05.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10972025-03-20T22:01:05.244ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-20 21:51:05.238845974 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10982025-03-20T22:01:05.244ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.789365ms iteration = 11 status = {"expiration":"2025-03-20T21:51:05.238845974Z","pruned":[]}
10992025-03-20T22:01:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11002025-03-20T22:01:05.246ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11012025-03-20T22:01:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11022025-03-20T22:01:05.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11032025-03-20T22:01:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11042025-03-20T22:01:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11052025-03-20T22:01:05.248ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11062025-03-20T22:01:05.249ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11072025-03-20T22:01:05.249ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11082025-03-20T22:01:05.249ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
11092025-03-20T22:01:05.249ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11102025-03-20T22:01:05.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11112025-03-20T22:01:05.250ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11122025-03-20T22:01:05.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
11132025-03-20T22:01:05.253ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
11142025-03-20T22:01:05.253ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11152025-03-20T22:01:05.253ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11162025-03-20T22:01:05.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
11172025-03-20T22:01:05.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11182025-03-20T22:01:05.254ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11192025-03-20T22:01:05.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11202025-03-20T22:01:05.254ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11212025-03-20T22:01:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11222025-03-20T22:01:05.254ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11232025-03-20T22:01:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11242025-03-20T22:01:05.257ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.338971ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
11252025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
11262025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11272025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
11282025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11292025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11302025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
11312025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11322025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11332025-03-20T22:01:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11342025-03-20T22:01:05.260ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11352025-03-20T22:01:05.260ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11362025-03-20T22:01:05.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11372025-03-20T22:01:05.261ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11382025-03-20T22:01:05.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11392025-03-20T22:01:05.263ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11402025-03-20T22:01:05.263ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11412025-03-20T22:01:05.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11422025-03-20T22:01:05.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11432025-03-20T22:01:05.264ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11442025-03-20T22:01:05.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11452025-03-20T22:01:05.265ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11462025-03-20T22:01:05.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11472025-03-20T22:01:05.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11482025-03-20T22:01:05.266ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11492025-03-20T22:01:05.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11502025-03-20T22:01:05.267ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11512025-03-20T22:01:05.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11522025-03-20T22:01:05.269ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3609, tv_nsec: 74874719 } })
11532025-03-20T22:01:05.269ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3609, tv_nsec: 75070534 } })
11542025-03-20T22:01:05.269ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
11552025-03-20T22:01:05.269ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
11562025-03-20T22:01:05.270ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11572025-03-20T22:01:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11582025-03-20T22:01:05.271ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 32.583232ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
11592025-03-20T22:01:05.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11602025-03-20T22:01:05.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11612025-03-20T22:01:05.274ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11622025-03-20T22:01:05.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11632025-03-20T22:01:05.275ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(bf24b433-f221-4272-a85b-9d4dd23b4c8c, 0), (9c109989-2af8-4193-8741-7e47315b5f03, 0), (7090a6c9-9a07-4d1f-8577-9c9b11dc226f, 0), (ec95e359-0018-4170-a505-aaf4897aeb64, 0)] collection_id = bf24b433-f221-4272-a85b-9d4dd23b4c8c
11642025-03-20T22:01:05.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11652025-03-20T22:01:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11662025-03-20T22:01:05.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11672025-03-20T22:01:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11682025-03-20T22:01:05.283ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11692025-03-20T22:01:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11702025-03-20T22:01:05.287ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3609, tv_nsec: 92848890 } })
11712025-03-20T22:01:05.287ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
11722025-03-20T22:01:05.293ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11732025-03-20T22:01:05.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11742025-03-20T22:01:05.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11752025-03-20T22:01:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11762025-03-20T22:01:05.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772025-03-20T22:01:05.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11782025-03-20T22:01:05.307ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
11792025-03-20T22:01:05.307ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
11802025-03-20T22:01:05.307ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
11812025-03-20T22:01:05.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11822025-03-20T22:01:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11832025-03-20T22:01:05.310ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11842025-03-20T22:01:05.311ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 72.438137ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
11852025-03-20T22:01:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11862025-03-20T22:01:05.318ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11872025-03-20T22:01:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11882025-03-20T22:01:05.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11892025-03-20T22:01:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
11902025-03-20T22:01:05.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11912025-03-20T22:01:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11922025-03-20T22:01:05.330ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:438
11932025-03-20T22:01:05.330ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 91.173808ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
11942025-03-20T22:01:05.334ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
11952025-03-20T22:01:05.334ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
11962025-03-20T22:01:05.334ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 95.019722ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
11972025-03-20T22:01:05.334ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11982025-03-20T22:01:05.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11992025-03-20T22:01:05.335ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
12002025-03-20T22:01:05.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
12012025-03-20T22:01:05.336ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12022025-03-20T22:01:05.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12032025-03-20T22:01:05.341ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12042025-03-20T22:01:05.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12052025-03-20T22:01:05.341ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 102.880332ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12062025-03-20T22:01:05.343ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12072025-03-20T22:01:05.344ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 105.292787ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12082025-03-20T22:01:05.344ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12092025-03-20T22:01:05.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12102025-03-20T22:01:05.345ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12112025-03-20T22:01:05.345ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
12122025-03-20T22:01:05.345ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12132025-03-20T22:01:05.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12142025-03-20T22:01:05.348ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12152025-03-20T22:01:05.348ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12162025-03-20T22:01:05.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12172025-03-20T22:01:05.349ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
12182025-03-20T22:01:05.349ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
12192025-03-20T22:01:05.349ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 110.377952ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
12202025-03-20T22:01:05.351ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12212025-03-20T22:01:05.353ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
12222025-03-20T22:01:05.353ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 115.043997ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
12232025-03-20T22:01:05.354ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12242025-03-20T22:01:05.355ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
12252025-03-20T22:01:05.355ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12262025-03-20T22:01:05.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12272025-03-20T22:01:05.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12282025-03-20T22:01:05.356ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12292025-03-20T22:01:05.356ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 117.104572ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
12302025-03-20T22:01:05.366ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12312025-03-20T22:01:05.369ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12322025-03-20T22:01:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12332025-03-20T22:01:05.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12342025-03-20T22:01:05.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12352025-03-20T22:01:05.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12362025-03-20T22:01:05.372ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12372025-03-20T22:01:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12382025-03-20T22:01:05.373ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
12392025-03-20T22:01:05.373ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
12402025-03-20T22:01:05.373ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
12412025-03-20T22:01:05.373ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12422025-03-20T22:01:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12432025-03-20T22:01:05.400ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
12442025-03-20T22:01:05.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
12452025-03-20T22:01:05.424ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
12462025-03-20T22:01:05.424ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12472025-03-20T22:01:05.424ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "79cb598c-5899-4c50-bcb0-3d01c6d737bd", "content-length": "26", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
12482025-03-20T22:01:05.424ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12492025-03-20T22:01:05.427ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12502025-03-20T22:01:05.432ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12512025-03-20T22:01:05.437ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
12522025-03-20T22:01:05.437ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12532025-03-20T22:01:05.438ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f571de03-cf02-4784-b74b-92023c12192d", "content-length": "26", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
12542025-03-20T22:01:05.438ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12552025-03-20T22:01:05.438ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12562025-03-20T22:01:05.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12572025-03-20T22:01:05.439ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12582025-03-20T22:01:05.452ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = bf24b433-f221-4272-a85b-9d4dd23b4c8c file = nexus/db-queries/src/db/datastore/inventory.rs:1442 ncabooses = 0 nclickhouse_keeper_membership = 0 ncollections = 0 ndatasets = 0 nerrors = 0 nnics = 0 nnvme_disk_firmware = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agent_zones = 0 nsled_agents = 0 nsps = 0 nzones = 0 nzpools = 0
12592025-03-20T22:01:05.452ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12602025-03-20T22:01:05.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
12612025-03-20T22:01:05.496ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
12622025-03-20T22:01:05.496ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12632025-03-20T22:01:05.496ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "99822000-c80e-404e-aaa8-6fc5b3e8b59b", "content-length": "26", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
12642025-03-20T22:01:05.496ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12652025-03-20T22:01:05.498ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
12662025-03-20T22:01:05.566ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
12672025-03-20T22:01:05.570ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
12682025-03-20T22:01:05.570ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12692025-03-20T22:01:05.570ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "359490fd-6ea8-4bf8-ab52-494ff183f886", "content-length": "372", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
12702025-03-20T22:01:05.571ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12712025-03-20T22:01:05.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12722025-03-20T22:01:05.575ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 336.346036ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12732025-03-20T22:01:05.579ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12742025-03-20T22:01:05.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12752025-03-20T22:01:05.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12762025-03-20T22:01:05.582ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12772025-03-20T22:01:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12782025-03-20T22:01:05.584ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
12792025-03-20T22:01:05.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12802025-03-20T22:01:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12812025-03-20T22:01:05.588ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 350.209566ms iteration = 23 status = {"physical_disks_added":0}
12822025-03-20T22:01:05.598ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12832025-03-20T22:01:05.610ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12842025-03-20T22:01:05.611ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2a4e06d-b4b1-481b-b78a-c64e9220e778", "content-length": "1", "date": "Thu, 20 Mar 2025 22:01:04 GMT"} })
12852025-03-20T22:01:05.613ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12862025-03-20T22:01:05.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12872025-03-20T22:01:05.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12882025-03-20T22:01:05.615ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12892025-03-20T22:01:05.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12902025-03-20T22:01:05.616ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 377.101696ms iteration = 21 status = null
12912025-03-20T22:01:05.618ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(9c109989-2af8-4193-8741-7e47315b5f03, 0), (7090a6c9-9a07-4d1f-8577-9c9b11dc226f, 0), (ec95e359-0018-4170-a505-aaf4897aeb64, 0)]
12922025-03-20T22:01:05.618ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3609, tv_nsec: 424435708 } })
12932025-03-20T22:01:05.620ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 381.551575ms iteration = 21 status = {}
12942025-03-20T22:01:05.627ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12952025-03-20T22:01:05.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12962025-03-20T22:01:05.629ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12972025-03-20T22:01:05.629ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12982025-03-20T22:01:05.629ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 390.872894ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
12992025-03-20T22:01:05.641ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13002025-03-20T22:01:05.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
13012025-03-20T22:01:05.647ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13022025-03-20T22:01:05.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13032025-03-20T22:01:05.648ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13042025-03-20T22:01:05.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13052025-03-20T22:01:05.650ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 411.567838ms iteration = 21 status = 0
13062025-03-20T22:01:05.652ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13072025-03-20T22:01:05.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13082025-03-20T22:01:05.653ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
13092025-03-20T22:01:05.653ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
13102025-03-20T22:01:05.653ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13112025-03-20T22:01:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13122025-03-20T22:01:05.660ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13132025-03-20T22:01:05.660ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13142025-03-20T22:01:05.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13152025-03-20T22:01:05.663ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13162025-03-20T22:01:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13172025-03-20T22:01:05.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13182025-03-20T22:01:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13192025-03-20T22:01:05.666ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13202025-03-20T22:01:05.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13212025-03-20T22:01:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
13222025-03-20T22:01:05.667ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13232025-03-20T22:01:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13242025-03-20T22:01:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13252025-03-20T22:01:05.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13262025-03-20T22:01:05.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13272025-03-20T22:01:05.668ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13282025-03-20T22:01:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13292025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13302025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
13312025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
13322025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-20 21:51:06.492803 UTC dns_group = internal generation = 1
13332025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 434.168022ms iteration = 11 status = {"generation":1}
13342025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13352025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13362025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13372025-03-20T22:01:05.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13382025-03-20T22:01:05.674ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13392025-03-20T22:01:05.676ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13402025-03-20T22:01:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13412025-03-20T22:01:05.682ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
13422025-03-20T22:01:05.682ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13432025-03-20T22:01:05.683ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bfb78237-0b76-4026-852c-caaaa1c1fa44", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
13442025-03-20T22:01:05.683ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13452025-03-20T22:01:05.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13462025-03-20T22:01:05.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13472025-03-20T22:01:05.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13482025-03-20T22:01:05.687ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13492025-03-20T22:01:05.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13502025-03-20T22:01:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13512025-03-20T22:01:05.694ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13522025-03-20T22:01:05.694ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13532025-03-20T22:01:05.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13542025-03-20T22:01:05.695ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13552025-03-20T22:01:05.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13562025-03-20T22:01:05.698ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13572025-03-20T22:01:05.698ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13582025-03-20T22:01:05.699ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "48d473a2-56df-4e1f-91bb-49ecab30bdce", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
13592025-03-20T22:01:05.701ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 462.191492ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
13602025-03-20T22:01:05.702ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 463.301102ms iteration = 21 status = {}
13612025-03-20T22:01:05.721ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
13622025-03-20T22:01:05.721ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 0 version = 2
13632025-03-20T22:01:05.721ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13642025-03-20T22:01:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13652025-03-20T22:01:05.724ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13662025-03-20T22:01:05.728ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13672025-03-20T22:01:05.728ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13682025-03-20T22:01:05.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13692025-03-20T22:01:05.730ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 1 version = 2
13702025-03-20T22:01:05.730ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
13712025-03-20T22:01:05.730ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-20 21:51:07.691085 UTC dns_group = external generation = 2
13722025-03-20T22:01:05.730ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 491.597268ms iteration = 11 status = {"generation":2}
13732025-03-20T22:01:05.734ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13742025-03-20T22:01:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13752025-03-20T22:01:05.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13762025-03-20T22:01:05.737ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13772025-03-20T22:01:05.738ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
13782025-03-20T22:01:05.738ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 499.070278ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
13792025-03-20T22:01:05.739ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13802025-03-20T22:01:05.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13812025-03-20T22:01:05.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13822025-03-20T22:01:05.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13832025-03-20T22:01:05.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13842025-03-20T22:01:05.742ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13852025-03-20T22:01:05.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13862025-03-20T22:01:05.744ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13872025-03-20T22:01:05.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13882025-03-20T22:01:05.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13892025-03-20T22:01:05.748ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13902025-03-20T22:01:05.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13912025-03-20T22:01:05.757ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
13922025-03-20T22:01:05.757ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
13932025-03-20T22:01:05.757ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
13942025-03-20T22:01:05.758ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e73d2729-7251-436e-815e-fdc62622f5c5", "content-length": "476", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
13952025-03-20T22:01:05.758ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
13962025-03-20T22:01:05.758ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-20T21:51:01.993052Z, time_modified: 2025-03-20T21:51:01.993052Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }) }
13972025-03-20T22:01:05.758ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 519.386642ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]}},"default_endpoint":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
13982025-03-20T22:01:05.758ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa5168c8-3b6b-4a04-a32b-ac415356a066", "content-length": "734", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
13992025-03-20T22:01:05.758ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
14002025-03-20T22:01:05.759ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01d4279d-132a-425f-abca-09ee9c5c4807", "content-length": "111", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14012025-03-20T22:01:05.759ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
14022025-03-20T22:01:05.759ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bb9a7f9-9fe7-4e3f-a6b1-ccf38760d358", "content-length": "111", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14032025-03-20T22:01:05.759ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
14042025-03-20T22:01:05.760ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d390f18f-6a8d-413f-b621-f5ed169c672d", "content-length": "167", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14052025-03-20T22:01:05.760ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
14062025-03-20T22:01:05.760ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7901b19e-d18c-4d33-907e-892af7ba6178", "content-length": "167", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14072025-03-20T22:01:05.760ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
14082025-03-20T22:01:05.760ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f41bc3b-fe92-4404-9cd9-ec42d0c9df0c", "content-length": "175", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14092025-03-20T22:01:05.760ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
14102025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd6892f0-acbb-41f6-89b4-284c92689597", "content-length": "175", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14112025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
14122025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f06dda14-6ee7-4803-92c5-d5a7d05bd1f8", "content-length": "702", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14132025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14142025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "df8d1384-ccd8-4bd9-8588-e8c7544fa968", "content-length": "727", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14152025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14162025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1b69c38-2e27-4135-a452-6d5b2e46562e", "content-length": "729", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14172025-03-20T22:01:05.761ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14182025-03-20T22:01:05.762ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ecb1e4-b097-4865-840a-ec5946dc5b1b", "content-length": "728", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14192025-03-20T22:01:05.762ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
14202025-03-20T22:01:05.762ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1942052-6c74-409b-8836-adbc7f3106ef", "content-length": "722", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14212025-03-20T22:01:05.762ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
14222025-03-20T22:01:05.762ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "86063423-2e17-4520-bc57-870cfc70cd52", "content-length": "109", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14232025-03-20T22:01:05.762ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
14242025-03-20T22:01:05.763ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "01ea22f3-bfbe-4379-9350-d655d6224546", "content-length": "109", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14252025-03-20T22:01:05.763ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
14262025-03-20T22:01:05.763ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2350801-ec33-4cfd-a73d-1de2117dc4ca", "content-length": "169", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14272025-03-20T22:01:05.763ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
14282025-03-20T22:01:05.763ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "192e474a-31f7-4e3f-b0bb-1d6db94add08", "content-length": "169", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14292025-03-20T22:01:05.763ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
14302025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "271460e3-e332-4fce-bcbb-e518d4246111", "content-length": "178", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14312025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
14322025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51e4008e-77ed-4613-b773-2d68dc832c77", "content-length": "178", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14332025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
14342025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "abb18934-a676-4340-a89e-d85de9d1e77a", "content-length": "702", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14352025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
14362025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfddfb1d-607f-4314-bfc4-9d34206b01dc", "content-length": "727", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14372025-03-20T22:01:05.764ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
14382025-03-20T22:01:05.765ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "82b375a7-08ee-4199-bf20-3885a9b54841", "content-length": "729", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14392025-03-20T22:01:05.765ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
14402025-03-20T22:01:05.765ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2253deb1-4f1e-4582-b027-3665b5b5a697", "content-length": "728", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
14412025-03-20T22:01:05.765ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14422025-03-20T22:01:05.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14432025-03-20T22:01:05.767ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14442025-03-20T22:01:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14452025-03-20T22:01:05.771ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
14462025-03-20T22:01:05.771ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 532.502192ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
14472025-03-20T22:01:05.853ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
14482025-03-20T22:01:05.853ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6e0a84ae-6b32-4bcf-8596-a88dd8e178e0", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
14492025-03-20T22:01:05.853ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
14502025-03-20T22:01:05.855ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "718a9469-6da3-4132-9eec-56d1b09afb00", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
14512025-03-20T22:01:05.855ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
14522025-03-20T22:01:05.858ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
14532025-03-20T22:01:05.858ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 619.427518ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
14542025-03-20T22:01:05.860ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14552025-03-20T22:01:05.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14562025-03-20T22:01:05.863ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14572025-03-20T22:01:05.863ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
14582025-03-20T22:01:05.863ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14592025-03-20T22:01:05.863ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 624.961915ms iteration = 5 status = {"blueprint_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","inventory_collection_id":"ec95e359-0018-4170-a505-aaf4897aeb64","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
14602025-03-20T22:01:05.864ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14612025-03-20T22:01:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14622025-03-20T22:01:05.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
14632025-03-20T22:01:05.866ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14642025-03-20T22:01:05.866ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14652025-03-20T22:01:05.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14662025-03-20T22:01:05.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
14672025-03-20T22:01:05.866ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14682025-03-20T22:01:05.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14692025-03-20T22:01:05.867ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
14702025-03-20T22:01:05.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14712025-03-20T22:01:05.867ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "09c7d2cc-af8d-41e9-b16b-6e7e34f183c9", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
14722025-03-20T22:01:05.867ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
14732025-03-20T22:01:05.868ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742025-03-20T22:01:05.869ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
14752025-03-20T22:01:05.869ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14762025-03-20T22:01:05.869ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ff833187-d771-4990-b80c-95a9da10d787", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
14772025-03-20T22:01:05.869ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
14782025-03-20T22:01:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
14792025-03-20T22:01:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14802025-03-20T22:01:05.871ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "0225a319-77b7-46ec-a86a-667937acc394", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
14812025-03-20T22:01:05.871ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
14822025-03-20T22:01:05.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
14832025-03-20T22:01:05.871ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14842025-03-20T22:01:05.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
14852025-03-20T22:01:05.873ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14862025-03-20T22:01:05.873ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
14872025-03-20T22:01:05.873ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 634.961563ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
14882025-03-20T22:01:05.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14892025-03-20T22:01:05.875ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14902025-03-20T22:01:05.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14912025-03-20T22:01:05.877ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14922025-03-20T22:01:05.879ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14932025-03-20T22:01:05.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14942025-03-20T22:01:05.882ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
14952025-03-20T22:01:05.882ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14962025-03-20T22:01:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14972025-03-20T22:01:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
14982025-03-20T22:01:05.882ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14992025-03-20T22:01:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15002025-03-20T22:01:05.884ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15012025-03-20T22:01:05.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15022025-03-20T22:01:05.887ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15032025-03-20T22:01:05.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15042025-03-20T22:01:05.889ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15052025-03-20T22:01:05.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15062025-03-20T22:01:05.892ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15072025-03-20T22:01:05.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15082025-03-20T22:01:05.894ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15092025-03-20T22:01:05.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15102025-03-20T22:01:05.897ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15112025-03-20T22:01:05.897ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "798c2e26-4023-4f48-a122-4c42b80e29c3", "content-length": "134", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
15122025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
15132025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
15142025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
15152025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
15162025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15172025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15182025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15192025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15202025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
15212025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
15222025-03-20T22:01:05.898ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15232025-03-20T22:01:05.898ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
15242025-03-20T22:01:05.898ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15252025-03-20T22:01:05.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15262025-03-20T22:01:05.901ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
15272025-03-20T22:01:05.901ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15282025-03-20T22:01:05.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15292025-03-20T22:01:05.901ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15302025-03-20T22:01:05.901ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "59a74fc9-29c5-4576-b45d-921a269b7c17", "content-length": "2", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
15312025-03-20T22:01:05.901ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15322025-03-20T22:01:05.901ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1e5553b-8b74-48ed-b51b-a713d27963ad", "content-length": "52", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
15332025-03-20T22:01:05.901ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
15342025-03-20T22:01:05.901ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15352025-03-20T22:01:05.903ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15362025-03-20T22:01:05.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
15372025-03-20T22:01:05.905ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15382025-03-20T22:01:05.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15392025-03-20T22:01:05.911ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15402025-03-20T22:01:05.913ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
15412025-03-20T22:01:05.913ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
15422025-03-20T22:01:05.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
15432025-03-20T22:01:05.918ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
15442025-03-20T22:01:05.935ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15452025-03-20T22:01:05.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15462025-03-20T22:01:05.971ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15472025-03-20T22:01:05.975ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15482025-03-20T22:01:05.975ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15492025-03-20T22:01:05.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15502025-03-20T22:01:05.977ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15512025-03-20T22:01:05.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15522025-03-20T22:01:05.978ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15532025-03-20T22:01:05.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15542025-03-20T22:01:05.994ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
15552025-03-20T22:01:05.994ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
15562025-03-20T22:01:05.996ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15572025-03-20T22:01:06.002ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
15582025-03-20T22:01:06.002ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15592025-03-20T22:01:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15602025-03-20T22:01:06.015ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15612025-03-20T22:01:06.021ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15622025-03-20T22:01:06.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15632025-03-20T22:01:06.023ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15642025-03-20T22:01:06.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15652025-03-20T22:01:06.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15662025-03-20T22:01:06.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15672025-03-20T22:01:06.028ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15682025-03-20T22:01:06.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15692025-03-20T22:01:06.029ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "29d03720-a268-49aa-bb33-3dcc1746f441", "date": "Thu, 20 Mar 2025 22:01:05 GMT"} })
15702025-03-20T22:01:06.029ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
15712025-03-20T22:01:06.029ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15722025-03-20T22:01:06.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b730fa92-075f-4eb1-93f1-c21bce155302 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
15732025-03-20T22:01:06.030ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b730fa92-075f-4eb1-93f1-c21bce155302 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
15742025-03-20T22:01:06.030ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd2fe337-2244-4b12-9f85-99593a2ead91", "content-length": "682", "date": "Thu, 20 Mar 2025 22:01:06 GMT"} })
15752025-03-20T22:01:06.030ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15762025-03-20T22:01:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15772025-03-20T22:01:06.030ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "93eca338-756f-4a99-a461-7bd2e94e756e", "content-length": "691", "date": "Thu, 20 Mar 2025 22:01:06 GMT"} })
15782025-03-20T22:01:06.030ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
15792025-03-20T22:01:06.030ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
15802025-03-20T22:01:06.030ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
15812025-03-20T22:01:06.031ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "e9090522-60b2-4f87-a2e3-f689106905cb", "date": "Thu, 20 Mar 2025 22:01:06 GMT"} })
15822025-03-20T22:01:06.031ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
15832025-03-20T22:01:06.031ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15842025-03-20T22:01:06.031ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "f523abf6-eac4-4d42-af7e-ac7dc0c33c09", "content-length": "26", "date": "Thu, 20 Mar 2025 22:01:06 GMT"} })
15852025-03-20T22:01:06.031ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:673 sled = http://[fd00:1122:3344:101::1]:12345
15862025-03-20T22:01:06.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15872025-03-20T22:01:06.032ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2451 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b730fa92-075f-4eb1-93f1-c21bce155302 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
15882025-03-20T22:01:06.033ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 793.955036ms iteration = 21 status = {}
15892025-03-20T22:01:06.033ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 794.060819ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":2,"put_config_err":0,"put_config_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"317fd31f-9b13-492c-9d67-8468dff1ea05","time":"2025-03-20T21:56:05.737132002Z"},{"operation":"list","target_sled":"317fd31f-9b13-492c-9d67-8468dff1ea05","time":"2025-03-20T21:56:05.737435370Z"},{"generation":1,"operation":"put_config","target_sled":"317fd31f-9b13-492c-9d67-8468dff1ea05","time":"2025-03-20T22:01:06.031102355Z"},{"operation":"list","target_sled":"317fd31f-9b13-492c-9d67-8468dff1ea05","time":"2025-03-20T22:01:06.031469225Z"}]}
15902025-03-20T22:01:06.034ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15912025-03-20T22:01:06.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15922025-03-20T22:01:06.038ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15932025-03-20T22:01:06.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15942025-03-20T22:01:06.040ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15952025-03-20T22:01:06.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15962025-03-20T22:01:06.048ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 809.467381ms iteration = 21 status = {}
15972025-03-20T22:01:06.069ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
15982025-03-20T22:01:06.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
15992025-03-20T22:01:06.071ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16002025-03-20T22:01:06.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16012025-03-20T22:01:06.073ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16022025-03-20T22:01:06.077ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16032025-03-20T22:01:06.077ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16042025-03-20T22:01:06.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16052025-03-20T22:01:06.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16062025-03-20T22:01:06.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16072025-03-20T22:01:06.080ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16082025-03-20T22:01:06.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16092025-03-20T22:01:06.082ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16102025-03-20T22:01:06.086ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16112025-03-20T22:01:06.086ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16122025-03-20T22:01:06.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16132025-03-20T22:01:06.093ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 0a978ab1-777d-4b0c-bcb4-e8f8837fc8e0, kind: Service { id: d3d1b222-fdac-4068-90a2-d02f8327ce99 }, name: Name("nexus-d3d1b222-fdac-4068-90a2-d02f8327ce99"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 91])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 2b5bf83f-e5c4-41cc-a71c-c3a9a3ee9963, kind: Service { id: 8602d1a8-380a-4598-9f5c-9268e05f383a }, name: Name("nexus-8602d1a8-380a-4598-9f5c-9268e05f383a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6fd19c19-eedd-4e0e-94ab-3e42b4d9f374, kind: Service { id: 6b5e38bb-80ea-495a-ac1b-7f1037ba058f }, name: Name("nexus-6b5e38bb-80ea-495a-ac1b-7f1037ba058f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 216])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 1e74727c-4bac-454e-bfe8-36f5c3e61c8b, kind: Service { id: 55400d62-c4af-483b-8223-1234d14cf0ad }, name: Name("external-dns-55400d62-c4af-483b-8223-1234d14cf0ad"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 186])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c06b1c3e-a7ff-48c0-8384-bfd512419736, kind: Service { id: d88b8f1a-4a6a-4e0b-ac74-40a39aba4124 }, name: Name("external-dns-d88b8f1a-4a6a-4e0b-ac74-40a39aba4124"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 57])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
16142025-03-20T22:01:06.093ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16152025-03-20T22:01:06.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16162025-03-20T22:01:06.094ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16172025-03-20T22:01:06.096ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
16182025-03-20T22:01:06.096ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16192025-03-20T22:01:06.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16202025-03-20T22:01:06.098ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
16212025-03-20T22:01:06.098ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1e74727c-4bac-454e-bfe8-36f5c3e61c8b, kind: Service { id: 55400d62-c4af-483b-8223-1234d14cf0ad }, name: Name("external-dns-55400d62-c4af-483b-8223-1234d14cf0ad"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 186])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c06b1c3e-a7ff-48c0-8384-bfd512419736, kind: Service { id: d88b8f1a-4a6a-4e0b-ac74-40a39aba4124 }, name: Name("external-dns-d88b8f1a-4a6a-4e0b-ac74-40a39aba4124"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 57])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 0a978ab1-777d-4b0c-bcb4-e8f8837fc8e0, kind: Service { id: d3d1b222-fdac-4068-90a2-d02f8327ce99 }, name: Name("nexus-d3d1b222-fdac-4068-90a2-d02f8327ce99"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 91])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 2b5bf83f-e5c4-41cc-a71c-c3a9a3ee9963, kind: Service { id: 8602d1a8-380a-4598-9f5c-9268e05f383a }, name: Name("nexus-8602d1a8-380a-4598-9f5c-9268e05f383a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 61])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6fd19c19-eedd-4e0e-94ab-3e42b4d9f374, kind: Service { id: 6b5e38bb-80ea-495a-ac1b-7f1037ba058f }, name: Name("nexus-6b5e38bb-80ea-495a-ac1b-7f1037ba058f"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 216])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
16222025-03-20T22:01:06.098ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
16232025-03-20T22:01:06.230ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 317fd31f-9b13-492c-9d67-8468dff1ea05, time_created: 2025-03-20T21:51:05.487567Z, time_modified: 2025-03-20T21:51:05.487567Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348) }]
16242025-03-20T22:01:06.230ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
16252025-03-20T22:01:06.230ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16262025-03-20T22:01:06.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16272025-03-20T22:01:06.233ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16282025-03-20T22:01:06.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16292025-03-20T22:01:06.234ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16302025-03-20T22:01:06.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
16312025-03-20T22:01:06.273ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16322025-03-20T22:01:06.278ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "cfa3e32a-e448-4b8b-aa85-b2927834225a", "date": "Thu, 20 Mar 2025 22:01:06 GMT"} })
16332025-03-20T22:01:06.279ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
16342025-03-20T22:01:06.279ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 1.040073539s
16352025-03-20T22:01:06.279ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 1.04008432s iteration = 3 status = {}
16362025-03-20T22:01:06.513ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5487f360-3d47-4c0f-b995-357fe2121ded", "content-length": "31969", "date": "Thu, 20 Mar 2025 22:01:06 GMT"} })
16372025-03-20T22:01:06.515ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16382025-03-20T22:01:06.515ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16392025-03-20T22:01:06.515ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
16402025-03-20T22:01:06.516ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16412025-03-20T22:01:06.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16422025-03-20T22:01:06.518ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16432025-03-20T22:01:06.520ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
16442025-03-20T22:01:06.520ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16452025-03-20T22:01:06.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
16462025-03-20T22:01:06.712ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 4b80742e-2a2e-418e-ad10-4bb85bb97e3b file = nexus/db-queries/src/db/datastore/inventory.rs:1010
16472025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 4b80742e-2a2e-418e-ad10-4bb85bb97e3b time_started = 2025-03-20 22:01:05.757860 UTC
16482025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Dependency
16492025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.474119308s iteration = 3 status = {"collection_id":"4b80742e-2a2e-418e-ad10-4bb85bb97e3b","time_done":"2025-03-20 22:01:06.515083 UTC","time_started":"2025-03-20 22:01:05.757860 UTC"}
16502025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16512025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
16522025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16532025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16542025-03-20T22:01:06.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16552025-03-20T22:01:06.715ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16562025-03-20T22:01:06.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16572025-03-20T22:01:06.717ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16582025-03-20T22:01:06.717ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16592025-03-20T22:01:06.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16602025-03-20T22:01:06.719ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
16612025-03-20T22:01:06.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16622025-03-20T22:01:06.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16632025-03-20T22:01:06.720ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16642025-03-20T22:01:06.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16652025-03-20T22:01:06.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16662025-03-20T22:01:06.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16672025-03-20T22:01:06.721ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16682025-03-20T22:01:06.723ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16692025-03-20T22:01:06.723ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16702025-03-20T22:01:06.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16712025-03-20T22:01:06.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16722025-03-20T22:01:06.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16732025-03-20T22:01:06.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16742025-03-20T22:01:06.755ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16752025-03-20T22:01:06.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16762025-03-20T22:01:06.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16772025-03-20T22:01:06.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16782025-03-20T22:01:06.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16792025-03-20T22:01:06.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16802025-03-20T22:01:06.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16812025-03-20T22:01:06.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16822025-03-20T22:01:06.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16832025-03-20T22:01:06.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16842025-03-20T22:01:06.768ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16852025-03-20T22:01:06.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16862025-03-20T22:01:06.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16872025-03-20T22:01:06.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16882025-03-20T22:01:06.774ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16892025-03-20T22:01:06.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16902025-03-20T22:01:06.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16912025-03-20T22:01:06.777ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
16922025-03-20T22:01:06.777ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 64.30305ms iteration = 6 status = {"blueprint_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","inventory_collection_id":"4b80742e-2a2e-418e-ad10-4bb85bb97e3b","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
16932025-03-20T22:01:06.827ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 114.360258ms iteration = 24 status = {"physical_disks_added":0}
16942025-03-20T22:01:07.072ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e30dd7e1-1bde-4b41-b8f9-abfbe0e5d619 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
16952025-03-20T22:01:07.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e30dd7e1-1bde-4b41-b8f9-abfbe0e5d619 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
16962025-03-20T22:01:07.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2558 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e30dd7e1-1bde-4b41-b8f9-abfbe0e5d619 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
16972025-03-20T22:01:07.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 545 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 0f3facfe-1dac-4eee-9c3b-ef0c347a0531 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
16982025-03-20T22:01:08.095ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 82218c78-7c50-44e8-a5d5-f2f16f4fed4b roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
16992025-03-20T22:01:08.095ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 82218c78-7c50-44e8-a5d5-f2f16f4fed4b resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17002025-03-20T22:01:08.097ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1926 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 82218c78-7c50-44e8-a5d5-f2f16f4fed4b response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17012025-03-20T22:01:09.116ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 59def859-bf37-4485-b9e4-17323e16ad7a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17022025-03-20T22:01:09.116ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 59def859-bf37-4485-b9e4-17323e16ad7a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17032025-03-20T22:01:09.118ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2104 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 59def859-bf37-4485-b9e4-17323e16ad7a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17042025-03-20T22:01:10.137ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6356469d-6f05-4fa6-8889-9a023d82b9f0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17052025-03-20T22:01:10.137ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6356469d-6f05-4fa6-8889-9a023d82b9f0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17062025-03-20T22:01:10.139ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2559 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6356469d-6f05-4fa6-8889-9a023d82b9f0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17072025-03-20T22:01:11.159ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fb2cc22f-7f04-4258-938b-61a0caa0a75f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17082025-03-20T22:01:11.159ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fb2cc22f-7f04-4258-938b-61a0caa0a75f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17092025-03-20T22:01:11.161ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2545 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fb2cc22f-7f04-4258-938b-61a0caa0a75f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17102025-03-20T22:01:12.182ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a9b21c32-b68c-42dc-89c8-5e66f92ff5d0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17112025-03-20T22:01:12.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a9b21c32-b68c-42dc-89c8-5e66f92ff5d0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17122025-03-20T22:01:12.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2295 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a9b21c32-b68c-42dc-89c8-5e66f92ff5d0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17132025-03-20T22:01:13.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8e6265dd-7679-4a46-963e-04ba987e2b96 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17142025-03-20T22:01:13.203ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8e6265dd-7679-4a46-963e-04ba987e2b96 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17152025-03-20T22:01:13.205ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2447 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8e6265dd-7679-4a46-963e-04ba987e2b96 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17162025-03-20T22:01:14.226ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9185c8a4-03f2-40fd-8a6f-297bc758cec7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17172025-03-20T22:01:14.226ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9185c8a4-03f2-40fd-8a6f-297bc758cec7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17182025-03-20T22:01:14.228ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2422 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9185c8a4-03f2-40fd-8a6f-297bc758cec7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17192025-03-20T22:01:15.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
17202025-03-20T22:01:15.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
17212025-03-20T22:01:15.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17222025-03-20T22:01:15.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17232025-03-20T22:01:15.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17242025-03-20T22:01:15.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17252025-03-20T22:01:15.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17262025-03-20T22:01:15.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17272025-03-20T22:01:15.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17282025-03-20T22:01:15.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17292025-03-20T22:01:15.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17302025-03-20T22:01:15.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17312025-03-20T22:01:15.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17322025-03-20T22:01:15.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 68fa0d1a-4271-4cae-b7ce-8f15b2f9fbe3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17332025-03-20T22:01:15.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 68fa0d1a-4271-4cae-b7ce-8f15b2f9fbe3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17342025-03-20T22:01:15.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
17352025-03-20T22:01:15.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17362025-03-20T22:01:15.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1584 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 68fa0d1a-4271-4cae-b7ce-8f15b2f9fbe3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17372025-03-20T22:01:15.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17382025-03-20T22:01:15.258ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
17392025-03-20T22:01:15.258ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.062339ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
17402025-03-20T22:01:15.267ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17412025-03-20T22:01:15.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17422025-03-20T22:01:15.298ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.143002ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17432025-03-20T22:01:16.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1e73590e-1404-4bfd-acee-ca663ff62230 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17442025-03-20T22:01:16.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1e73590e-1404-4bfd-acee-ca663ff62230 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17452025-03-20T22:01:16.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1518 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1e73590e-1404-4bfd-acee-ca663ff62230 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17462025-03-20T22:01:17.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 643 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 2db1ff09-db34-4c87-8c1b-334e6f3c2d35 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
17472025-03-20T22:01:17.290ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 32aa055e-539f-40d9-8d98-fe7af2687cb4 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17482025-03-20T22:01:17.290ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 32aa055e-539f-40d9-8d98-fe7af2687cb4 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17492025-03-20T22:01:17.292ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2411 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 32aa055e-539f-40d9-8d98-fe7af2687cb4 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17502025-03-20T22:01:18.313ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 298b1e1c-1f33-4383-a5cf-9a739adfb030 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17512025-03-20T22:01:18.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 298b1e1c-1f33-4383-a5cf-9a739adfb030 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17522025-03-20T22:01:18.315ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2164 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 298b1e1c-1f33-4383-a5cf-9a739adfb030 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17532025-03-20T22:01:19.335ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ea7451e7-756b-48a5-a3a1-111a90ecdcc2 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17542025-03-20T22:01:19.336ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ea7451e7-756b-48a5-a3a1-111a90ecdcc2 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17552025-03-20T22:01:19.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1604 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ea7451e7-756b-48a5-a3a1-111a90ecdcc2 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17562025-03-20T22:01:20.358ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a224cf97-0c8d-4d21-a901-6915f5670613 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17572025-03-20T22:01:20.359ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a224cf97-0c8d-4d21-a901-6915f5670613 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17582025-03-20T22:01:20.361ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2208 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a224cf97-0c8d-4d21-a901-6915f5670613 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17592025-03-20T22:01:21.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1c9d549a-3511-4db0-892d-5a8d30d99b43 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17602025-03-20T22:01:21.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1c9d549a-3511-4db0-892d-5a8d30d99b43 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17612025-03-20T22:01:21.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2568 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1c9d549a-3511-4db0-892d-5a8d30d99b43 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17622025-03-20T22:01:22.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 71314660-c532-4bcf-b3bd-1bb869ba8977 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17632025-03-20T22:01:22.403ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 71314660-c532-4bcf-b3bd-1bb869ba8977 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17642025-03-20T22:01:22.405ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2114 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 71314660-c532-4bcf-b3bd-1bb869ba8977 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17652025-03-20T22:01:23.424ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 575e8e21-7330-4641-b65b-c5247cbe5c22 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17662025-03-20T22:01:23.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 575e8e21-7330-4641-b65b-c5247cbe5c22 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17672025-03-20T22:01:23.426ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2580 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 575e8e21-7330-4641-b65b-c5247cbe5c22 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17682025-03-20T22:01:24.445ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c9b8e01-71f4-4604-897d-96576e8e2b1e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17692025-03-20T22:01:24.445ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c9b8e01-71f4-4604-897d-96576e8e2b1e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17702025-03-20T22:01:24.447ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1992 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c9b8e01-71f4-4604-897d-96576e8e2b1e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17712025-03-20T22:01:25.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
17722025-03-20T22:01:25.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17732025-03-20T22:01:25.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
17742025-03-20T22:01:25.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17752025-03-20T22:01:25.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17762025-03-20T22:01:25.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17772025-03-20T22:01:25.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17782025-03-20T22:01:25.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17792025-03-20T22:01:25.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17802025-03-20T22:01:25.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17812025-03-20T22:01:25.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17822025-03-20T22:01:25.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17832025-03-20T22:01:25.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17842025-03-20T22:01:25.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
17852025-03-20T22:01:25.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17862025-03-20T22:01:25.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17872025-03-20T22:01:25.260ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
17882025-03-20T22:01:25.260ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.286326ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
17892025-03-20T22:01:25.269ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17902025-03-20T22:01:25.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17912025-03-20T22:01:25.299ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 61.135286ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17922025-03-20T22:01:25.466ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fdb2d4e6-c8ec-4008-b220-0adcfc48a232 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17932025-03-20T22:01:25.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fdb2d4e6-c8ec-4008-b220-0adcfc48a232 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17942025-03-20T22:01:25.468ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1417 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fdb2d4e6-c8ec-4008-b220-0adcfc48a232 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17952025-03-20T22:01:26.488ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 11a57996-d023-4cd5-b29d-1b229f6ff9ba roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17962025-03-20T22:01:26.488ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 11a57996-d023-4cd5-b29d-1b229f6ff9ba resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17972025-03-20T22:01:26.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1503 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 11a57996-d023-4cd5-b29d-1b229f6ff9ba response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
17982025-03-20T22:01:27.255ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 809 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 2c764952-8b8b-47e8-9419-72c7e08e4178 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
17992025-03-20T22:01:27.511ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 666b66b4-0167-4b79-b897-48787480330a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18002025-03-20T22:01:27.511ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 666b66b4-0167-4b79-b897-48787480330a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18012025-03-20T22:01:27.513ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2006 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 666b66b4-0167-4b79-b897-48787480330a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18022025-03-20T22:01:28.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 378b0574-0b6e-4638-87bf-ac7214cdbbbd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18032025-03-20T22:01:28.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 378b0574-0b6e-4638-87bf-ac7214cdbbbd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18042025-03-20T22:01:28.535ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1720 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 378b0574-0b6e-4638-87bf-ac7214cdbbbd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18052025-03-20T22:01:29.555ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ea26da39-3a0c-477b-af17-5fca30b668dd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18062025-03-20T22:01:29.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ea26da39-3a0c-477b-af17-5fca30b668dd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18072025-03-20T22:01:29.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2145 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ea26da39-3a0c-477b-af17-5fca30b668dd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18082025-03-20T22:01:30.577ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53c9fe31-2104-4769-9ce2-089f05ebdb9e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18092025-03-20T22:01:30.577ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53c9fe31-2104-4769-9ce2-089f05ebdb9e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18102025-03-20T22:01:30.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2270 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53c9fe31-2104-4769-9ce2-089f05ebdb9e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18112025-03-20T22:01:31.600ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 94f9db31-f95e-4a43-b5fd-0559a78b3b7b roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18122025-03-20T22:01:31.601ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 94f9db31-f95e-4a43-b5fd-0559a78b3b7b resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18132025-03-20T22:01:31.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2354 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 94f9db31-f95e-4a43-b5fd-0559a78b3b7b response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18142025-03-20T22:01:32.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 171f2ea6-53c2-4424-8c77-c40974e2f789 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18152025-03-20T22:01:32.622ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 171f2ea6-53c2-4424-8c77-c40974e2f789 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18162025-03-20T22:01:32.624ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1939 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 171f2ea6-53c2-4424-8c77-c40974e2f789 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18172025-03-20T22:01:33.644ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c59b1403-9e6c-4a8b-8818-5d1551e83351 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18182025-03-20T22:01:33.644ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c59b1403-9e6c-4a8b-8818-5d1551e83351 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18192025-03-20T22:01:33.646ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2103 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c59b1403-9e6c-4a8b-8818-5d1551e83351 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18202025-03-20T22:01:34.666ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 150b695a-ad60-4eff-aff9-03fd6b8e5da6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18212025-03-20T22:01:34.666ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 150b695a-ad60-4eff-aff9-03fd6b8e5da6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18222025-03-20T22:01:34.668ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2067 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 150b695a-ad60-4eff-aff9-03fd6b8e5da6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
18232025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
18242025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
18252025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18262025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
18272025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18282025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
18292025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
18302025-03-20T22:01:35.238ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
18312025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18322025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
18332025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18342025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18352025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
18362025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18372025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
18382025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18392025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
18402025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
18412025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
18422025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
18432025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18442025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
18452025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18462025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
18472025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
18482025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
18492025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18502025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
18512025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18522025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
18532025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18542025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
18552025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
18562025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
18572025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
18582025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18592025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
18602025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18612025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
18622025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18632025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18642025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18652025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18662025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18672025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18682025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18692025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18702025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18712025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
18722025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18732025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
18742025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
18752025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18762025-03-20T22:01:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18772025-03-20T22:01:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
18782025-03-20T22:01:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
18792025-03-20T22:01:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18802025-03-20T22:01:35.240ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18812025-03-20T22:01:35.244ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18822025-03-20T22:01:35.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
18832025-03-20T22:01:35.249ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18842025-03-20T22:01:35.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18852025-03-20T22:01:35.251ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 13.227543ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
18862025-03-20T22:01:35.252ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3639, tv_nsec: 57770401 } })
18872025-03-20T22:01:35.252ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18882025-03-20T22:01:35.252ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
18892025-03-20T22:01:35.252ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18902025-03-20T22:01:35.252ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18912025-03-20T22:01:35.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18922025-03-20T22:01:35.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
18932025-03-20T22:01:35.255ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18942025-03-20T22:01:35.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18952025-03-20T22:01:35.260ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18962025-03-20T22:01:35.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18972025-03-20T22:01:35.262ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18982025-03-20T22:01:35.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18992025-03-20T22:01:35.266ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3639, tv_nsec: 71812487 } })
19002025-03-20T22:01:35.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19012025-03-20T22:01:35.266ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19022025-03-20T22:01:35.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19032025-03-20T22:01:35.266ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
19042025-03-20T22:01:35.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19052025-03-20T22:01:35.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19062025-03-20T22:01:35.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19072025-03-20T22:01:35.274ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19082025-03-20T22:01:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19092025-03-20T22:01:35.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19102025-03-20T22:01:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19112025-03-20T22:01:35.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19122025-03-20T22:01:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19132025-03-20T22:01:35.281ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19142025-03-20T22:01:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19152025-03-20T22:01:35.295ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19162025-03-20T22:01:35.295ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
19172025-03-20T22:01:35.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19182025-03-20T22:01:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19192025-03-20T22:01:35.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19202025-03-20T22:01:35.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19212025-03-20T22:01:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19222025-03-20T22:01:35.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19232025-03-20T22:01:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19242025-03-20T22:01:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
19252025-03-20T22:01:35.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19262025-03-20T22:01:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19272025-03-20T22:01:35.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.71556ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
19282025-03-20T22:01:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19292025-03-20T22:01:35.325ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3639, tv_nsec: 131581846 } })
19302025-03-20T22:01:35.326ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
19312025-03-20T22:01:35.326ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19322025-03-20T22:01:35.326ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19332025-03-20T22:01:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19342025-03-20T22:01:35.327ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19352025-03-20T22:01:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
19362025-03-20T22:01:35.327ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
19372025-03-20T22:01:35.327ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 89.675168ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
19382025-03-20T22:01:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
19392025-03-20T22:01:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19402025-03-20T22:01:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19412025-03-20T22:01:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
19422025-03-20T22:01:35.329ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19432025-03-20T22:01:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19442025-03-20T22:01:35.330ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19452025-03-20T22:01:35.333ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19462025-03-20T22:01:35.333ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 95.389521ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
19472025-03-20T22:01:35.335ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 96.779339ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
19482025-03-20T22:01:35.335ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 97.372784ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19492025-03-20T22:01:35.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
19502025-03-20T22:01:35.336ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19512025-03-20T22:01:35.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19522025-03-20T22:01:35.336ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19532025-03-20T22:01:35.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19542025-03-20T22:01:35.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19552025-03-20T22:01:35.338ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19562025-03-20T22:01:35.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19572025-03-20T22:01:35.340ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
19582025-03-20T22:01:35.340ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19592025-03-20T22:01:35.341ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6881557-61e2-414b-b954-58a0a14f60d8", "content-length": "26", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
19602025-03-20T22:01:35.341ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19612025-03-20T22:01:35.343ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19622025-03-20T22:01:35.348ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
19632025-03-20T22:01:35.348ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 109.632992ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
19642025-03-20T22:01:35.358ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19652025-03-20T22:01:35.358ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
19662025-03-20T22:01:35.358ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19672025-03-20T22:01:35.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19682025-03-20T22:01:35.363ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19692025-03-20T22:01:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19702025-03-20T22:01:35.366ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19712025-03-20T22:01:35.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19722025-03-20T22:01:35.367ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 128.99089ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19732025-03-20T22:01:35.368ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
19742025-03-20T22:01:35.368ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19752025-03-20T22:01:35.368ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2f97889-35df-495a-b401-423f296dacf1", "content-length": "26", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
19762025-03-20T22:01:35.368ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19772025-03-20T22:01:35.368ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19782025-03-20T22:01:35.370ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19792025-03-20T22:01:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
19802025-03-20T22:01:35.372ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19812025-03-20T22:01:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19822025-03-20T22:01:35.384ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19832025-03-20T22:01:35.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19842025-03-20T22:01:35.384ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 146.500808ms iteration = 22 status = 0
19852025-03-20T22:01:35.385ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
19862025-03-20T22:01:35.385ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 147.131906ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
19872025-03-20T22:01:35.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 147.885245ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19882025-03-20T22:01:35.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 148.152823ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
19892025-03-20T22:01:35.410ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19902025-03-20T22:01:35.410ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
19912025-03-20T22:01:35.410ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 172.125134ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
19922025-03-20T22:01:35.414ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19932025-03-20T22:01:35.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19942025-03-20T22:01:35.430ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 192.207012ms iteration = 25 status = {"physical_disks_added":0}
19952025-03-20T22:01:35.485ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 247.102979ms iteration = 22 status = {}
19962025-03-20T22:01:35.497ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
19972025-03-20T22:01:35.497ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19982025-03-20T22:01:35.497ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0e3e5ec-5a91-4315-8231-05a82c1da5c9", "content-length": "26", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
19992025-03-20T22:01:35.497ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20002025-03-20T22:01:35.499ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
20012025-03-20T22:01:35.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20022025-03-20T22:01:35.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20032025-03-20T22:01:35.525ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20042025-03-20T22:01:35.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20052025-03-20T22:01:35.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20062025-03-20T22:01:35.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20072025-03-20T22:01:35.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20082025-03-20T22:01:35.531ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
20092025-03-20T22:01:35.531ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20102025-03-20T22:01:35.531ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7f8ac8b-5149-4dc7-a587-9644b67cdaaa", "content-length": "372", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20112025-03-20T22:01:35.531ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20122025-03-20T22:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20132025-03-20T22:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20142025-03-20T22:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20152025-03-20T22:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20162025-03-20T22:01:35.533ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
20172025-03-20T22:01:35.534ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "18eff2bf-0319-4a9f-a822-767e32590dd2", "content-length": "1", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20182025-03-20T22:01:35.535ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20192025-03-20T22:01:35.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20202025-03-20T22:01:35.539ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 301.145855ms iteration = 22 status = null
20212025-03-20T22:01:35.558ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
20222025-03-20T22:01:35.558ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
20232025-03-20T22:01:35.558ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1ca033c5-f7d4-4f09-aee8-608087f997f4", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20242025-03-20T22:01:35.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20252025-03-20T22:01:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20262025-03-20T22:01:35.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20272025-03-20T22:01:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20282025-03-20T22:01:35.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20292025-03-20T22:01:35.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20302025-03-20T22:01:35.567ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
20312025-03-20T22:01:35.567ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20322025-03-20T22:01:35.567ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7eeb7236-c7eb-4176-8b7b-7378d7c88eb6", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20332025-03-20T22:01:35.569ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 330.872419ms iteration = 22 status = {}
20342025-03-20T22:01:35.638ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20352025-03-20T22:01:35.641ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
20362025-03-20T22:01:35.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20372025-03-20T22:01:35.644ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
20382025-03-20T22:01:35.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20392025-03-20T22:01:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20402025-03-20T22:01:35.654ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
20412025-03-20T22:01:35.655ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a79c9070-2f64-400c-9e2b-334f9ac956ee", "content-length": "134", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20422025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
20432025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
20442025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
20452025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
20462025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20472025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20482025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20492025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20502025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20512025-03-20T22:01:35.655ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20522025-03-20T22:01:35.656ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
20532025-03-20T22:01:35.656ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20542025-03-20T22:01:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20552025-03-20T22:01:35.658ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
20562025-03-20T22:01:35.658ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "996589bd-5a89-4870-9944-acde004fa982", "content-length": "2", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20572025-03-20T22:01:35.658ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
20582025-03-20T22:01:35.659ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5875197-f9c9-4e2d-88e2-4859f7fa1254", "content-length": "52", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20592025-03-20T22:01:35.659ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
20602025-03-20T22:01:35.659ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20612025-03-20T22:01:35.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 55814951-8c6f-466a-b0e8-a6f6e81852b5 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20622025-03-20T22:01:35.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 55814951-8c6f-466a-b0e8-a6f6e81852b5 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20632025-03-20T22:01:35.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2010 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 55814951-8c6f-466a-b0e8-a6f6e81852b5 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20642025-03-20T22:01:35.717ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "27a4ee4d-5e9f-401d-89c4-7ac89c672424", "content-length": "691", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20652025-03-20T22:01:35.717ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
20662025-03-20T22:01:35.717ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
20672025-03-20T22:01:35.717ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
20682025-03-20T22:01:35.719ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 480.997145ms iteration = 22 status = {}
20692025-03-20T22:01:35.800ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2bde2531-a66f-4ec3-bb30-2bb1506f6426", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20702025-03-20T22:01:35.800ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
20712025-03-20T22:01:35.800ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20722025-03-20T22:01:35.801ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a9a349d-7be2-4aec-b1ec-59417416e35f", "content-length": "682", "date": "Thu, 20 Mar 2025 22:01:35 GMT"} })
20732025-03-20T22:01:35.801ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20742025-03-20T22:01:35.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20752025-03-20T22:01:35.804ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20762025-03-20T22:01:35.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20772025-03-20T22:01:35.808ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20782025-03-20T22:01:35.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20792025-03-20T22:01:35.810ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20802025-03-20T22:01:35.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20812025-03-20T22:01:35.818ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 579.867619ms iteration = 22 status = {}
20822025-03-20T22:01:36.732ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d308b521-be55-47dd-b437-38b5fab88187 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20832025-03-20T22:01:36.732ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d308b521-be55-47dd-b437-38b5fab88187 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20842025-03-20T22:01:36.733ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1607 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d308b521-be55-47dd-b437-38b5fab88187 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20852025-03-20T22:01:37.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 558 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 4e020fce-6154-4871-8f3d-d6d7ef9b6c52 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
20862025-03-20T22:01:37.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bc13d840-fd25-408a-b719-a1df1c773ff1 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20872025-03-20T22:01:37.752ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bc13d840-fd25-408a-b719-a1df1c773ff1 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20882025-03-20T22:01:37.754ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2407 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bc13d840-fd25-408a-b719-a1df1c773ff1 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20892025-03-20T22:01:38.775ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 071d7b23-b5a4-4413-b24c-17ee9d1940c6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20902025-03-20T22:01:38.776ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 071d7b23-b5a4-4413-b24c-17ee9d1940c6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20912025-03-20T22:01:38.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2712 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 071d7b23-b5a4-4413-b24c-17ee9d1940c6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20922025-03-20T22:01:39.797ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cc42939e-2c89-42a3-985a-6e8785562bf2 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20932025-03-20T22:01:39.798ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cc42939e-2c89-42a3-985a-6e8785562bf2 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20942025-03-20T22:01:39.799ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1939 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cc42939e-2c89-42a3-985a-6e8785562bf2 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20952025-03-20T22:01:40.820ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8968c483-ccd2-4580-9eed-14dbecce2b87 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20962025-03-20T22:01:40.820ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8968c483-ccd2-4580-9eed-14dbecce2b87 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20972025-03-20T22:01:40.823ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2841 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8968c483-ccd2-4580-9eed-14dbecce2b87 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20982025-03-20T22:01:41.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6bc3c272-80e0-4b53-84ee-f738766d9a48 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
20992025-03-20T22:01:41.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6bc3c272-80e0-4b53-84ee-f738766d9a48 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21002025-03-20T22:01:41.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2232 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6bc3c272-80e0-4b53-84ee-f738766d9a48 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21012025-03-20T22:01:42.873ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 614ab254-d2a8-4134-82f8-51b3cfa8f652 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21022025-03-20T22:01:42.873ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 614ab254-d2a8-4134-82f8-51b3cfa8f652 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21032025-03-20T22:01:42.875ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2345 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 614ab254-d2a8-4134-82f8-51b3cfa8f652 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21042025-03-20T22:01:43.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ce9bd788-db63-44b0-b749-12ff7d71bcb3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21052025-03-20T22:01:43.896ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ce9bd788-db63-44b0-b749-12ff7d71bcb3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21062025-03-20T22:01:43.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2320 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ce9bd788-db63-44b0-b749-12ff7d71bcb3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21072025-03-20T22:01:44.918ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9167320f-c621-4424-bb1b-11ca677d7e1f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21082025-03-20T22:01:44.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9167320f-c621-4424-bb1b-11ca677d7e1f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21092025-03-20T22:01:44.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1960 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9167320f-c621-4424-bb1b-11ca677d7e1f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21102025-03-20T22:01:45.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
21112025-03-20T22:01:45.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21122025-03-20T22:01:45.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
21132025-03-20T22:01:45.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21142025-03-20T22:01:45.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21152025-03-20T22:01:45.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21162025-03-20T22:01:45.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21172025-03-20T22:01:45.241ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21182025-03-20T22:01:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21192025-03-20T22:01:45.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21202025-03-20T22:01:45.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21212025-03-20T22:01:45.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21222025-03-20T22:01:45.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21232025-03-20T22:01:45.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21242025-03-20T22:01:45.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21252025-03-20T22:01:45.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
21262025-03-20T22:01:45.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21272025-03-20T22:01:45.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21282025-03-20T22:01:45.264ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
21292025-03-20T22:01:45.264ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.04072ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
21302025-03-20T22:01:45.279ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.427511ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21312025-03-20T22:01:45.942ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7c466a0b-5a7b-435a-b603-fbe4705664c0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21322025-03-20T22:01:45.943ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7c466a0b-5a7b-435a-b603-fbe4705664c0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21332025-03-20T22:01:45.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1743 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7c466a0b-5a7b-435a-b603-fbe4705664c0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21342025-03-20T22:01:46.964ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d0829f5-addd-47a6-a0aa-e1d44dd977ca roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21352025-03-20T22:01:46.964ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d0829f5-addd-47a6-a0aa-e1d44dd977ca resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21362025-03-20T22:01:46.966ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1814 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d0829f5-addd-47a6-a0aa-e1d44dd977ca response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21372025-03-20T22:01:47.248ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:42088
21382025-03-20T22:01:47.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 2a2bacaf-41c1-4461-bc95-3689718b8230 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
21392025-03-20T22:01:47.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 2a2bacaf-41c1-4461-bc95-3689718b8230 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
21402025-03-20T22:01:47.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3175 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 2a2bacaf-41c1-4461-bc95-3689718b8230 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
21412025-03-20T22:01:47.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = a2b4cc91-5e88-4c34-a46d-c006f187e8a2 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
21422025-03-20T22:01:47.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = a2b4cc91-5e88-4c34-a46d-c006f187e8a2 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
21432025-03-20T22:01:47.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 532 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = e054d0e4-541e-4b1a-b2de-2cae578ade30 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
21442025-03-20T22:01:47.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2647 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = a2b4cc91-5e88-4c34-a46d-c006f187e8a2 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
21452025-03-20T22:01:47.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 16892dbf-7ca4-4960-bddc-b9d23b981a8f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21462025-03-20T22:01:47.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 16892dbf-7ca4-4960-bddc-b9d23b981a8f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21472025-03-20T22:01:47.988ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2433 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 16892dbf-7ca4-4960-bddc-b9d23b981a8f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21482025-03-20T22:01:49.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53c2850e-a15b-4b14-b4ce-e8c9d2e183b4 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21492025-03-20T22:01:49.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53c2850e-a15b-4b14-b4ce-e8c9d2e183b4 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21502025-03-20T22:01:49.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1753 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53c2850e-a15b-4b14-b4ce-e8c9d2e183b4 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21512025-03-20T22:01:50.033ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0789a9c-60c0-4d9c-816a-a3da722def6b roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21522025-03-20T22:01:50.033ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0789a9c-60c0-4d9c-816a-a3da722def6b resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21532025-03-20T22:01:50.035ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2095 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0789a9c-60c0-4d9c-816a-a3da722def6b response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21542025-03-20T22:01:51.055ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1bb54507-3ae1-4277-b5ca-6981ae2a4275 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21552025-03-20T22:01:51.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1bb54507-3ae1-4277-b5ca-6981ae2a4275 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21562025-03-20T22:01:51.057ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2100 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1bb54507-3ae1-4277-b5ca-6981ae2a4275 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21572025-03-20T22:01:52.077ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5d354c0d-0eee-4a1a-8f6d-c3b3e9e3ef40 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21582025-03-20T22:01:52.078ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5d354c0d-0eee-4a1a-8f6d-c3b3e9e3ef40 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21592025-03-20T22:01:52.080ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2495 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5d354c0d-0eee-4a1a-8f6d-c3b3e9e3ef40 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21602025-03-20T22:01:53.102ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80aeb436-281b-458d-944e-a2439e4a2661 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21612025-03-20T22:01:53.102ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80aeb436-281b-458d-944e-a2439e4a2661 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21622025-03-20T22:01:53.104ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2608 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80aeb436-281b-458d-944e-a2439e4a2661 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21632025-03-20T22:01:54.126ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bdf01630-ab65-420d-ae63-77f235f452d3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21642025-03-20T22:01:54.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bdf01630-ab65-420d-ae63-77f235f452d3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21652025-03-20T22:01:54.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2257 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bdf01630-ab65-420d-ae63-77f235f452d3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21662025-03-20T22:01:55.158ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 07ad28d1-369e-460c-b2d1-4dd2739f1aaa roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21672025-03-20T22:01:55.158ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 07ad28d1-369e-460c-b2d1-4dd2739f1aaa resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21682025-03-20T22:01:55.160ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2555 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 07ad28d1-369e-460c-b2d1-4dd2739f1aaa response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21692025-03-20T22:01:55.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
21702025-03-20T22:01:55.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
21712025-03-20T22:01:55.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21722025-03-20T22:01:55.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21732025-03-20T22:01:55.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21742025-03-20T22:01:55.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21752025-03-20T22:01:55.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21762025-03-20T22:01:55.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21772025-03-20T22:01:55.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21782025-03-20T22:01:55.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21792025-03-20T22:01:55.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21802025-03-20T22:01:55.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21812025-03-20T22:01:55.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21822025-03-20T22:01:55.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
21832025-03-20T22:01:55.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21842025-03-20T22:01:55.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21852025-03-20T22:01:55.260ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
21862025-03-20T22:01:55.260ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.093921ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
21872025-03-20T22:01:55.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21882025-03-20T22:01:55.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21892025-03-20T22:01:55.307ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.448701ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21902025-03-20T22:01:56.181ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4f6009ef-236b-47aa-99fd-ca3a9d6ce224 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21912025-03-20T22:01:56.182ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4f6009ef-236b-47aa-99fd-ca3a9d6ce224 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21922025-03-20T22:01:56.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2572 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4f6009ef-236b-47aa-99fd-ca3a9d6ce224 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21932025-03-20T22:01:57.205ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e19c472a-d678-49f5-9fda-7d339c4a1283 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21942025-03-20T22:01:57.205ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e19c472a-d678-49f5-9fda-7d339c4a1283 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21952025-03-20T22:01:57.207ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2228 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e19c472a-d678-49f5-9fda-7d339c4a1283 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21962025-03-20T22:01:57.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 768 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = d2f53f25-7e31-404e-bd4a-4cb4feb44723 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
21972025-03-20T22:01:58.228ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c79ca5d8-21a9-4840-b695-18c2cb2b5330 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21982025-03-20T22:01:58.228ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c79ca5d8-21a9-4840-b695-18c2cb2b5330 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
21992025-03-20T22:01:58.230ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c79ca5d8-21a9-4840-b695-18c2cb2b5330 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22002025-03-20T22:01:59.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b93ff776-bf57-4c66-a031-e52be95614d2 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22012025-03-20T22:01:59.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b93ff776-bf57-4c66-a031-e52be95614d2 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22022025-03-20T22:01:59.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2491 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b93ff776-bf57-4c66-a031-e52be95614d2 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22032025-03-20T22:01:59.366ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fdbe37bb-f303-46d6-bd63-7907762b9f2a uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-03-20T22:01:59.345318161Z }
22042025-03-20T22:01:59.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fdbe37bb-f303-46d6-bd63-7907762b9f2a roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
22052025-03-20T22:01:59.366ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fdbe37bb-f303-46d6-bd63-7907762b9f2a resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
22062025-03-20T22:01:59.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 36294 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fdbe37bb-f303-46d6-bd63-7907762b9f2a response_code = 204 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
22072025-03-20T22:01:59.505ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c31649af-fe5c-48b2-b362-e6002241de65 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-20T22:01:59.504744814Z }
22082025-03-20T22:01:59.505ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c31649af-fe5c-48b2-b362-e6002241de65 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
22092025-03-20T22:01:59.505ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c31649af-fe5c-48b2-b362-e6002241de65 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
22102025-03-20T22:01:59.520ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 15054 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c31649af-fe5c-48b2-b362-e6002241de65 response_code = 204 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
22112025-03-20T22:02:00.275ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a55f7414-eabb-4499-b8df-0eb63c3e84a3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22122025-03-20T22:02:00.275ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a55f7414-eabb-4499-b8df-0eb63c3e84a3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22132025-03-20T22:02:00.277ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2527 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a55f7414-eabb-4499-b8df-0eb63c3e84a3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22142025-03-20T22:02:01.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1cc3c319-c0a1-4f55-bb07-b0cb6895b2ba roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22152025-03-20T22:02:01.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1cc3c319-c0a1-4f55-bb07-b0cb6895b2ba resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22162025-03-20T22:02:01.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2121 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1cc3c319-c0a1-4f55-bb07-b0cb6895b2ba response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22172025-03-20T22:02:02.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 7b2ded6a-8aa1-4b99-99ee-c06183bdd4c9 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
22182025-03-20T22:02:02.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 7b2ded6a-8aa1-4b99-99ee-c06183bdd4c9 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
22192025-03-20T22:02:02.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3716 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 7b2ded6a-8aa1-4b99-99ee-c06183bdd4c9 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
22202025-03-20T22:02:02.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b964efe4-5356-4de4-9072-99ef040d462e roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
22212025-03-20T22:02:02.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b964efe4-5356-4de4-9072-99ef040d462e resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
22222025-03-20T22:02:02.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2801 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b964efe4-5356-4de4-9072-99ef040d462e response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
22232025-03-20T22:02:02.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 40d97969-3811-42f9-b9da-f0dd37ee1f52 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22242025-03-20T22:02:02.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 40d97969-3811-42f9-b9da-f0dd37ee1f52 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22252025-03-20T22:02:02.320ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2448 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 40d97969-3811-42f9-b9da-f0dd37ee1f52 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22262025-03-20T22:02:03.342ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5a568c69-289c-4728-8c04-9664c71eacce roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22272025-03-20T22:02:03.342ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5a568c69-289c-4728-8c04-9664c71eacce resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22282025-03-20T22:02:03.344ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2413 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5a568c69-289c-4728-8c04-9664c71eacce response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22292025-03-20T22:02:04.364ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 998c4359-dcfb-4090-877c-325f8357c3eb roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22302025-03-20T22:02:04.364ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 998c4359-dcfb-4090-877c-325f8357c3eb resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22312025-03-20T22:02:04.366ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2158 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 998c4359-dcfb-4090-877c-325f8357c3eb response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
22322025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
22332025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
22342025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22352025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22362025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
22372025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
22382025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22392025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
22402025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
22412025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
22422025-03-20T22:02:05.238ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f
22432025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22442025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.24\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f"}
22452025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
22462025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
22472025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
22482025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22492025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
22502025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
22512025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
22522025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22532025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
22542025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22552025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
22562025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
22572025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
22582025-03-20T22:02:05.238ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-20 21:52:05.238622940 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
22592025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
22602025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
22612025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22622025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
22632025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
22642025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
22652025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22662025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
22672025-03-20T22:02:05.238ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
22682025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
22692025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22702025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
22712025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
22722025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
22732025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22742025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
22752025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
22762025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
22772025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
22782025-03-20T22:02:05.238ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
22792025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22802025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
22812025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
22822025-03-20T22:02:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
22832025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
22842025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22852025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
22862025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
22872025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22882025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
22892025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
22902025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
22912025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
22922025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22932025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22942025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
22952025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
22962025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
22972025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22982025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22992025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23002025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
23012025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23022025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23032025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23042025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23052025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
23062025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23072025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
23082025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23092025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23102025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
23112025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
23122025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
23132025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23142025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23152025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
23162025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
23172025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23182025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23192025-03-20T22:02:05.239ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55400d62-c4af-483b-8223-1234d14cf0ad.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3669, tv_nsec: 45349839 } })
23202025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23212025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
23222025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23232025-03-20T22:02:05.239ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58723511-3cb5-4c26-bebf-16620d9cd641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("59acd018-3b3d-4cb6-bd0f-2dfaf2b8d965.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 3669, tv_nsec: 45408881 } })
23242025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23252025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
23262025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
23272025-03-20T22:02:05.239ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.289994ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
23282025-03-20T22:02:05.240ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
23292025-03-20T22:02:05.240ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.333005ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
23302025-03-20T22:02:05.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23312025-03-20T22:02:05.241ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23322025-03-20T22:02:05.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23332025-03-20T22:02:05.243ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.318336ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
23342025-03-20T22:02:05.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23352025-03-20T22:02:05.245ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23362025-03-20T22:02:05.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23372025-03-20T22:02:05.245ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3669, tv_nsec: 51782521 } })
23382025-03-20T22:02:05.246ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
23392025-03-20T22:02:05.258ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-20 21:52:05.238622940 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
23402025-03-20T22:02:05.258ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 19.378019ms iteration = 12 status = {"expiration":"2025-03-20T21:52:05.238622940Z","pruned":[]}
23412025-03-20T22:02:05.260ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3669, tv_nsec: 66271669 } })
23422025-03-20T22:02:05.260ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
23432025-03-20T22:02:05.270ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 32.027297ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
23442025-03-20T22:02:05.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23452025-03-20T22:02:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23462025-03-20T22:02:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
23472025-03-20T22:02:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
23482025-03-20T22:02:05.294ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 55.411332ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
23492025-03-20T22:02:05.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23502025-03-20T22:02:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23512025-03-20T22:02:05.308ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23522025-03-20T22:02:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23532025-03-20T22:02:05.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23542025-03-20T22:02:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23552025-03-20T22:02:05.316ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23562025-03-20T22:02:05.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23572025-03-20T22:02:05.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.592612ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
23582025-03-20T22:02:05.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23592025-03-20T22:02:05.320ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
23602025-03-20T22:02:05.320ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
23612025-03-20T22:02:05.321ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 82.517887ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
23622025-03-20T22:02:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
23632025-03-20T22:02:05.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23642025-03-20T22:02:05.321ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23652025-03-20T22:02:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23662025-03-20T22:02:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23672025-03-20T22:02:05.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 84.631174ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
23682025-03-20T22:02:05.323ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23692025-03-20T22:02:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
23702025-03-20T22:02:05.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23712025-03-20T22:02:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23722025-03-20T22:02:05.324ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23732025-03-20T22:02:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23742025-03-20T22:02:05.325ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 86.307488ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23752025-03-20T22:02:05.325ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23762025-03-20T22:02:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
23772025-03-20T22:02:05.325ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
23782025-03-20T22:02:05.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
23792025-03-20T22:02:05.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23802025-03-20T22:02:05.327ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23812025-03-20T22:02:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23822025-03-20T22:02:05.327ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23832025-03-20T22:02:05.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23842025-03-20T22:02:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23852025-03-20T22:02:05.329ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23862025-03-20T22:02:05.329ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23872025-03-20T22:02:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23882025-03-20T22:02:05.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23892025-03-20T22:02:05.330ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23902025-03-20T22:02:05.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23912025-03-20T22:02:05.332ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23922025-03-20T22:02:05.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23932025-03-20T22:02:05.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
23942025-03-20T22:02:05.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23952025-03-20T22:02:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23962025-03-20T22:02:05.333ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23972025-03-20T22:02:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23982025-03-20T22:02:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23992025-03-20T22:02:05.334ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24002025-03-20T22:02:05.334ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
24012025-03-20T22:02:05.334ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24022025-03-20T22:02:05.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24032025-03-20T22:02:05.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24042025-03-20T22:02:05.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24052025-03-20T22:02:05.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24062025-03-20T22:02:05.341ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24072025-03-20T22:02:05.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24082025-03-20T22:02:05.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24092025-03-20T22:02:05.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24102025-03-20T22:02:05.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24112025-03-20T22:02:05.343ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24122025-03-20T22:02:05.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24132025-03-20T22:02:05.345ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24142025-03-20T22:02:05.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24152025-03-20T22:02:05.348ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3669, tv_nsec: 154583021 } })
24162025-03-20T22:02:05.348ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
24172025-03-20T22:02:05.349ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24182025-03-20T22:02:05.349ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24192025-03-20T22:02:05.349ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
24202025-03-20T22:02:05.349ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24212025-03-20T22:02:05.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
24222025-03-20T22:02:05.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24232025-03-20T22:02:05.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24242025-03-20T22:02:05.350ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24252025-03-20T22:02:05.352ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24262025-03-20T22:02:05.352ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24272025-03-20T22:02:05.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24282025-03-20T22:02:05.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
24292025-03-20T22:02:05.354ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24302025-03-20T22:02:05.354ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
24312025-03-20T22:02:05.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24322025-03-20T22:02:05.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24332025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
24342025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24352025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24362025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
24372025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
24382025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
24392025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24402025-03-20T22:02:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24412025-03-20T22:02:05.366ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
24422025-03-20T22:02:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
24432025-03-20T22:02:05.368ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
24442025-03-20T22:02:05.368ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 129.491004ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
24452025-03-20T22:02:05.372ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
24462025-03-20T22:02:05.372ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
24472025-03-20T22:02:05.372ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
24482025-03-20T22:02:05.372ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24492025-03-20T22:02:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24502025-03-20T22:02:05.374ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24512025-03-20T22:02:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24522025-03-20T22:02:05.379ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
24532025-03-20T22:02:05.379ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
24542025-03-20T22:02:05.379ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 140.502818ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
24552025-03-20T22:02:05.382ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 144.510205ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
24562025-03-20T22:02:05.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 145.82454ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24572025-03-20T22:02:05.384ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
24582025-03-20T22:02:05.384ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
24592025-03-20T22:02:05.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24602025-03-20T22:02:05.385ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 146.715774ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
24612025-03-20T22:02:05.386ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fc457eca-d225-431b-b2f6-65a4a76fb5f4 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
24622025-03-20T22:02:05.386ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fc457eca-d225-431b-b2f6-65a4a76fb5f4 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
24632025-03-20T22:02:05.387ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24642025-03-20T22:02:05.387ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
24652025-03-20T22:02:05.387ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 148.168513ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
24662025-03-20T22:02:05.396ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 158.5068ms iteration = 26 status = {"physical_disks_added":0}
24672025-03-20T22:02:05.399ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24682025-03-20T22:02:05.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24692025-03-20T22:02:05.412ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
24702025-03-20T22:02:05.412ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24712025-03-20T22:02:05.412ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ee9a271-b199-4fd5-92d6-ab1f347a0df8", "content-length": "26", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
24722025-03-20T22:02:05.412ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24732025-03-20T22:02:05.413ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24742025-03-20T22:02:05.428ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
24752025-03-20T22:02:05.428ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24762025-03-20T22:02:05.428ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "125c2503-2b35-4dd1-bc40-0bb31dd3a6b0", "content-length": "26", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
24772025-03-20T22:02:05.428ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24782025-03-20T22:02:05.430ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24792025-03-20T22:02:05.453ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24802025-03-20T22:02:05.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24812025-03-20T22:02:05.521ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
24822025-03-20T22:02:05.521ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "98997440-dc11-4758-b9bd-372961b9c3f2", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24832025-03-20T22:02:05.521ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24842025-03-20T22:02:05.535ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e0debd54-ddcc-4470-a511-46f6ec117523", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24852025-03-20T22:02:05.535ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24862025-03-20T22:02:05.541ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
24872025-03-20T22:02:05.541ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24882025-03-20T22:02:05.542ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4a9c493-1424-44b4-9bde-f6cb715aecc4", "content-length": "26", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
24892025-03-20T22:02:05.542ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24902025-03-20T22:02:05.551ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24912025-03-20T22:02:05.551ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 312.844277ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
24922025-03-20T22:02:05.551ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
24932025-03-20T22:02:05.565ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
24942025-03-20T22:02:05.565ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24952025-03-20T22:02:05.567ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8d72e2c-45f3-4c61-afad-a1c4ec33fb01", "content-length": "583", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
24962025-03-20T22:02:05.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24972025-03-20T22:02:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24982025-03-20T22:02:05.592ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
24992025-03-20T22:02:05.617ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 378.464062ms iteration = 23 status = {}
25002025-03-20T22:02:05.657ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/b8a518ba-8675-4da7-b93b-394df66485d7/state
25012025-03-20T22:02:05.658ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/b8a518ba-8675-4da7-b93b-394df66485d7/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "441fc978-b8e1-43a2-8238-8772ac7158b4", "content-length": "130", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
25022025-03-20T22:02:05.660ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25032025-03-20T22:02:05.662ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25042025-03-20T22:02:05.662ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25052025-03-20T22:02:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25062025-03-20T22:02:05.663ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 277133 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fc457eca-d225-431b-b2f6-65a4a76fb5f4 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
25072025-03-20T22:02:05.663ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 sled_id = 317fd31f-9b13-492c-9d67-8468dff1ea05 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-20T22:01:59.504744814Z }, migration_in: None, migration_out: None } vmm_id = b8a518ba-8675-4da7-b93b-394df66485d7
25082025-03-20T22:02:05.663ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2088 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 sled_id = 317fd31f-9b13-492c-9d67-8468dff1ea05 vmm_id = b8a518ba-8675-4da7-b93b-394df66485d7 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-20T22:01:59.504744814Z }
25092025-03-20T22:02:05.663ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 roles = RoleSet { roles: {} } sled_id = 317fd31f-9b13-492c-9d67-8468dff1ea05 vmm_id = b8a518ba-8675-4da7-b93b-394df66485d7
25102025-03-20T22:02:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 resource = Database result = Ok(()) sled_id = 317fd31f-9b13-492c-9d67-8468dff1ea05 vmm_id = b8a518ba-8675-4da7-b93b-394df66485d7
25112025-03-20T22:02:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25122025-03-20T22:02:05.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25132025-03-20T22:02:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25142025-03-20T22:02:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
25152025-03-20T22:02:05.666ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25162025-03-20T22:02:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25172025-03-20T22:02:05.669ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25182025-03-20T22:02:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
25192025-03-20T22:02:05.673ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25202025-03-20T22:02:05.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25212025-03-20T22:02:05.675ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 437.221394ms iteration = 23 status = 0
25222025-03-20T22:02:05.692ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
25232025-03-20T22:02:05.692ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "2860c62f-2f6a-4946-935e-526109d30ad0", "content-length": "1", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
25242025-03-20T22:02:05.695ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25252025-03-20T22:02:05.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25262025-03-20T22:02:05.697ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 459.12314ms iteration = 23 status = null
25272025-03-20T22:02:05.699ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25282025-03-20T22:02:05.700ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
25292025-03-20T22:02:05.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25302025-03-20T22:02:05.702ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25312025-03-20T22:02:05.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25322025-03-20T22:02:05.702ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 464.007711ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
25332025-03-20T22:02:05.706ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
25342025-03-20T22:02:05.706ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "41968696-dfc4-42d7-a81e-f4c37f1c07cf", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
25352025-03-20T22:02:05.706ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
25362025-03-20T22:02:05.708ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2ee7e2a1-f711-4256-9ed8-e4ecddaaef44", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
25372025-03-20T22:02:05.708ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
25382025-03-20T22:02:05.710ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d29cd52b-af76-4e88-a1b4-1d178f452a45", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
25392025-03-20T22:02:05.710ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
25402025-03-20T22:02:05.711ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25412025-03-20T22:02:05.714ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
25422025-03-20T22:02:05.714ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25432025-03-20T22:02:05.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25442025-03-20T22:02:05.714ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
25452025-03-20T22:02:05.714ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 476.082154ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
25462025-03-20T22:02:05.715ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
25472025-03-20T22:02:05.715ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
25482025-03-20T22:02:05.715ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25492025-03-20T22:02:05.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25502025-03-20T22:02:05.716ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {ff319170-4baf-4895-8b2c-d98744adb3bf: {10.151.2.175: {3fc4079f-7589-4966-bd0c-4e8b47ccc186}, 10.151.2.176: {3fc4079f-7589-4966-bd0c-4e8b47ccc186}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
25512025-03-20T22:02:05.716ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
25522025-03-20T22:02:05.716ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "23ef1f2b-b661-4100-85f1-8d728428ede5", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
25532025-03-20T22:02:05.716ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25542025-03-20T22:02:05.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25552025-03-20T22:02:05.721ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25562025-03-20T22:02:05.722ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
25572025-03-20T22:02:05.722ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 483.944234ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
25582025-03-20T22:02:05.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25592025-03-20T22:02:05.724ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25602025-03-20T22:02:05.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25612025-03-20T22:02:05.725ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25622025-03-20T22:02:05.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25632025-03-20T22:02:05.729ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25642025-03-20T22:02:05.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25652025-03-20T22:02:05.733ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25662025-03-20T22:02:05.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25672025-03-20T22:02:05.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25682025-03-20T22:02:05.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25692025-03-20T22:02:05.739ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25702025-03-20T22:02:05.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25712025-03-20T22:02:05.743ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
25722025-03-20T22:02:05.743ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
25732025-03-20T22:02:05.743ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25742025-03-20T22:02:05.743ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "104817e7-42a5-4f5e-a057-9a6da7ce21e4", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
25752025-03-20T22:02:05.745ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25762025-03-20T22:02:05.745ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 506.97484ms iteration = 23 status = {}
25772025-03-20T22:02:05.747ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25782025-03-20T22:02:05.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25792025-03-20T22:02:05.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25802025-03-20T22:02:05.754ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25812025-03-20T22:02:05.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25822025-03-20T22:02:05.757ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25832025-03-20T22:02:05.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25842025-03-20T22:02:05.760ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
25852025-03-20T22:02:05.760ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
25862025-03-20T22:02:05.760ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-20 21:51:06.492803 UTC dns_group = internal generation = 1
25872025-03-20T22:02:05.760ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 521.240301ms iteration = 12 status = {"generation":1}
25882025-03-20T22:02:05.760ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
25892025-03-20T22:02:05.760ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 0 version = 2
25902025-03-20T22:02:05.760ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25912025-03-20T22:02:05.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25922025-03-20T22:02:05.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25932025-03-20T22:02:05.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25942025-03-20T22:02:05.764ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25952025-03-20T22:02:05.765ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-20T21:51:01.993052Z, time_modified: 2025-03-20T21:51:01.993052Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
25962025-03-20T22:02:05.765ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 526.475411ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]}},"default_endpoint":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
25972025-03-20T22:02:05.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25982025-03-20T22:02:05.767ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 1 version = 2
25992025-03-20T22:02:05.767ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
26002025-03-20T22:02:05.767ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-20 21:51:07.691085 UTC dns_group = external generation = 2
26012025-03-20T22:02:05.767ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 528.998029ms iteration = 12 status = {"generation":2}
26022025-03-20T22:02:05.769ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
26032025-03-20T22:02:05.769ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 530.735796ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
26042025-03-20T22:02:05.781ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26052025-03-20T22:02:05.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26062025-03-20T22:02:05.783ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
26072025-03-20T22:02:05.783ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26082025-03-20T22:02:05.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26092025-03-20T22:02:05.803ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
26102025-03-20T22:02:05.803ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c0d9d8c-a9dc-4cd0-b119-38ef472dce1f", "content-length": "134", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
26112025-03-20T22:02:05.803ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
26122025-03-20T22:02:05.803ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
26132025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
26142025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26152025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26162025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26172025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26182025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26192025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26202025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26212025-03-20T22:02:05.804ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
26222025-03-20T22:02:05.804ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26232025-03-20T22:02:05.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26242025-03-20T22:02:05.807ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
26252025-03-20T22:02:05.807ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "1874c77a-8426-4d60-88d9-eebdc3d7701c", "content-length": "2", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
26262025-03-20T22:02:05.807ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
26272025-03-20T22:02:05.807ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "239d473e-4bdc-419f-8d6a-8afc46369af2", "content-length": "52", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
26282025-03-20T22:02:05.807ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
26292025-03-20T22:02:05.807ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26302025-03-20T22:02:05.850ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
26312025-03-20T22:02:05.907ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "877c688b-ee9a-4e43-9df0-def4ca66adb0", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
26322025-03-20T22:02:05.907ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
26332025-03-20T22:02:05.907ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
26342025-03-20T22:02:05.908ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a016a4e-feec-41b0-bf05-48a01b2dc843", "content-length": "682", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
26352025-03-20T22:02:05.908ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26362025-03-20T22:02:05.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26372025-03-20T22:02:05.912ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "15d6400b-bb2f-491f-893a-75b26d3d7656", "content-length": "810", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
26382025-03-20T22:02:05.912ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(5536642) }} sled = sock
26392025-03-20T22:02:05.912ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
26402025-03-20T22:02:05.912ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
26412025-03-20T22:02:05.912ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26422025-03-20T22:02:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26432025-03-20T22:02:05.913ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 674.861461ms iteration = 23 status = {}
26442025-03-20T22:02:05.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26452025-03-20T22:02:05.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26462025-03-20T22:02:05.918ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26472025-03-20T22:02:05.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26482025-03-20T22:02:05.927ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 688.373872ms iteration = 23 status = {}
26492025-03-20T22:02:06.223ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
26502025-03-20T22:02:06.223ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
26512025-03-20T22:02:06.253ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
26522025-03-20T22:02:06.253ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:34933
26532025-03-20T22:02:06.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34933 req_id = df9efbd8-09c1-411e-aa42-c1af46f378bb roles = RoleSet { roles: {} } uri = /metrics/producers
26542025-03-20T22:02:06.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34933 req_id = df9efbd8-09c1-411e-aa42-c1af46f378bb resource = Database result = Ok(()) uri = /metrics/producers
26552025-03-20T22:02:06.288ZINFOnexus: assigned collector to new producer collector_id = c75d8631-e01d-47db-98de-f38076ccc33e file = nexus/src/app/oximeter.rs:76 producer_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
26562025-03-20T22:02:06.288ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 34985 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34933 req_id = df9efbd8-09c1-411e-aa42-c1af46f378bb response_code = 201 uri = /metrics/producers
26572025-03-20T22:02:06.289ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "df9efbd8-09c1-411e-aa42-c1af46f378bb", "content-length": "41", "date": "Thu, 20 Mar 2025 22:02:05 GMT"} })
26582025-03-20T22:02:06.291ZDEBGnexus (producer-registration-task): registered with nexus successfully
26592025-03-20T22:02:06.291ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
26602025-03-20T22:02:06.692ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62bd4a14-fe8f-44a7-bc16-687ad0c60ba6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26612025-03-20T22:02:06.692ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62bd4a14-fe8f-44a7-bc16-687ad0c60ba6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26622025-03-20T22:02:06.694ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2425 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62bd4a14-fe8f-44a7-bc16-687ad0c60ba6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26632025-03-20T22:02:07.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 700 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = a6016e7a-8c53-4562-8b8c-7b2d8dbf3629 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
26642025-03-20T22:02:07.714ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 158fd5c2-b38a-44b0-8108-9d7163410d93 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26652025-03-20T22:02:07.714ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 158fd5c2-b38a-44b0-8108-9d7163410d93 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26662025-03-20T22:02:07.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2193 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 158fd5c2-b38a-44b0-8108-9d7163410d93 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26672025-03-20T22:02:08.736ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = af462da7-c993-42e9-b5a4-b6b9337e12ac roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26682025-03-20T22:02:08.736ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = af462da7-c993-42e9-b5a4-b6b9337e12ac resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26692025-03-20T22:02:08.738ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2445 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = af462da7-c993-42e9-b5a4-b6b9337e12ac response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26702025-03-20T22:02:08.865ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 remote_addr = [fd00:1122:3344:101::1]:64383 req_id = eb1854f8-8db1-4b5e-9638-a62898bbbc67 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-03-20T22:02:08.864920304Z }
26712025-03-20T22:02:08.865ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = eb1854f8-8db1-4b5e-9638-a62898bbbc67 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26722025-03-20T22:02:08.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = eb1854f8-8db1-4b5e-9638-a62898bbbc67 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26732025-03-20T22:02:08.923ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 26 transaction = vmm_and_migration_update_runtime
26742025-03-20T22:02:08.958ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = vmm_and_migration_update_runtime
26752025-03-20T22:02:08.958ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 93450 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = eb1854f8-8db1-4b5e-9638-a62898bbbc67 response_code = 204 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26762025-03-20T22:02:09.759ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e2100dda-6fb1-4263-b744-db26f3b14d4c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26772025-03-20T22:02:09.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e2100dda-6fb1-4263-b744-db26f3b14d4c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26782025-03-20T22:02:09.762ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2695 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e2100dda-6fb1-4263-b744-db26f3b14d4c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26792025-03-20T22:02:10.782ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 276fbf50-e5fa-48e0-9bd7-b2572daee411 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26802025-03-20T22:02:10.782ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 276fbf50-e5fa-48e0-9bd7-b2572daee411 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26812025-03-20T22:02:10.784ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2295 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 276fbf50-e5fa-48e0-9bd7-b2572daee411 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
26822025-03-20T22:02:11.379ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-20T22:02:11.378880875Z }
26832025-03-20T22:02:11.379ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26842025-03-20T22:02:11.379ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26852025-03-20T22:02:11.422ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7 vmm_needs_update = true
26862025-03-20T22:02:11.422ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26872025-03-20T22:02:11.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26882025-03-20T22:02:11.427ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26892025-03-20T22:02:11.427ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26902025-03-20T22:02:11.430ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26912025-03-20T22:02:11.430ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26922025-03-20T22:02:11.435ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26932025-03-20T22:02:11.435ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26942025-03-20T22:02:11.437ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26952025-03-20T22:02:11.437ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26962025-03-20T22:02:11.439ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26972025-03-20T22:02:11.439ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26982025-03-20T22:02:11.468ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
26992025-03-20T22:02:11.473ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4841a72c-7688-4028-b721-8d89119ea996, lookup_type: ById(4841a72c-7688-4028-b721-8d89119ea996) }, key: e5716097-f669-4df9-b053-eb0ddb431a2d, lookup_type: ById(e5716097-f669-4df9-b053-eb0ddb431a2d) }, key: d594712d-5385-423f-a318-6ea315a0e8a5, lookup_type: ById(d594712d-5385-423f-a318-6ea315a0e8a5) } result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
27002025-03-20T22:02:11.481ZINFOnexus (dropshot_internal): starting update saga for d594712d-5385-423f-a318-6ea315a0e8a5 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1551 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-20T22:02:11.378880875Z }
27012025-03-20T22:02:11.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 101973 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 response_code = 204 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
27022025-03-20T22:02:11.481ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 90a79c3a-85a0-47d3-9172-407f30710710 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-20T22:02:11.378880875Z }
27032025-03-20T22:02:11.481ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 90a79c3a-85a0-47d3-9172-407f30710710 roles = RoleSet { roles: {} } uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
27042025-03-20T22:02:11.481ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 90a79c3a-85a0-47d3-9172-407f30710710 resource = Database result = Ok(()) uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
27052025-03-20T22:02:11.483ZDEBGnexus (dropshot_internal): preparing instance-update saga for d594712d-5385-423f-a318-6ea315a0e8a5... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae5424e0-0926-458b-b3f4-7b1119c4e442 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
27062025-03-20T22:02:11.483ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update
27072025-03-20T22:02:11.485ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4160 local_addr = [fd00:1122:3344:101::c]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 90a79c3a-85a0-47d3-9172-407f30710710 response_code = 204 uri = /vmms/b8a518ba-8675-4da7-b93b-394df66485d7
27082025-03-20T22:02:11.488ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"d594712d-5385-423f-a318-6ea315a0e8a5","lookup_type":{"ById":"d594712d-5385-423f-a318-6ea315a0e8a5"},"parent":{"key":"e5716097-f669-4df9-b053-eb0ddb431a2d","lookup_type":{"ById":"e5716097-f669-4df9-b053-eb0ddb431a2d"},"parent":{"key":"4841a72c-7688-4028-b721-8d89119ea996","lookup_type":{"ById":"4841a72c-7688-4028-b721-8d89119ea996"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27092025-03-20T22:02:11.488ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update
27102025-03-20T22:02:11.525ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update
27112025-03-20T22:02:11.525ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27122025-03-20T22:02:11.530ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27132025-03-20T22:02:11.530ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
27142025-03-20T22:02:11.537ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27152025-03-20T22:02:11.537ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
27162025-03-20T22:02:11.543ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27172025-03-20T22:02:11.543ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
27182025-03-20T22:02:11.549ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27192025-03-20T22:02:11.549ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("59bb6c15-43c8-49e4-a9c6-a9681155d1d3")) node_id = 0 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
27202025-03-20T22:02:11.555ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27212025-03-20T22:02:11.555ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
27222025-03-20T22:02:11.566ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update
27232025-03-20T22:02:11.571ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27242025-03-20T22:02:11.572ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27252025-03-20T22:02:11.574ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27262025-03-20T22:02:11.574ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27272025-03-20T22:02:11.576ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27282025-03-20T22:02:11.576ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27292025-03-20T22:02:11.585ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27302025-03-20T22:02:11.586ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27312025-03-20T22:02:11.587ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27322025-03-20T22:02:11.587ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27332025-03-20T22:02:11.589ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27342025-03-20T22:02:11.589ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27352025-03-20T22:02:11.590ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27362025-03-20T22:02:11.596ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4841a72c-7688-4028-b721-8d89119ea996, lookup_type: ById(4841a72c-7688-4028-b721-8d89119ea996) }, key: e5716097-f669-4df9-b053-eb0ddb431a2d, lookup_type: ById(e5716097-f669-4df9-b053-eb0ddb431a2d) }, key: d594712d-5385-423f-a318-6ea315a0e8a5, lookup_type: ById(d594712d-5385-423f-a318-6ea315a0e8a5) } result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27372025-03-20T22:02:11.596ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance updater_id = 59bb6c15-43c8-49e4-a9c6-a9681155d1d3
27382025-03-20T22:02:11.596ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27392025-03-20T22:02:11.596ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27402025-03-20T22:02:11.609ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance updater_id = 59bb6c15-43c8-49e4-a9c6-a9681155d1d3
27412025-03-20T22:02:11.609ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27422025-03-20T22:02:11.610ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance
27432025-03-20T22:02:11.616ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 locked_gen = Generation(Generation(2)) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = LockInstance updater_id = 59bb6c15-43c8-49e4-a9c6-a9681155d1d3
27442025-03-20T22:02:11.616ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27452025-03-20T22:02:11.616ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("59bb6c15-43c8-49e4-a9c6-a9681155d1d3")}) node_id = 1 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
27462025-03-20T22:02:11.620ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27472025-03-20T22:02:11.620ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
27482025-03-20T22:02:11.627ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27492025-03-20T22:02:11.628ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27502025-03-20T22:02:11.629ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27512025-03-20T22:02:11.629ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27522025-03-20T22:02:11.630ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27532025-03-20T22:02:11.631ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27542025-03-20T22:02:11.631ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27552025-03-20T22:02:11.635ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4841a72c-7688-4028-b721-8d89119ea996, lookup_type: ById(4841a72c-7688-4028-b721-8d89119ea996) }, key: e5716097-f669-4df9-b053-eb0ddb431a2d, lookup_type: ById(e5716097-f669-4df9-b053-eb0ddb431a2d) }, key: d594712d-5385-423f-a318-6ea315a0e8a5, lookup_type: ById(d594712d-5385-423f-a318-6ea315a0e8a5) } result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27562025-03-20T22:02:11.635ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27572025-03-20T22:02:11.635ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
27582025-03-20T22:02:11.645ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: b8a518ba-8675-4da7-b93b-394df66485d7, time_created: 2025-03-20T22:01:50.421365Z, time_deleted: None, instance_id: d594712d-5385-423f-a318-6ea315a0e8a5, sled_id: 317fd31f-9b13-492c-9d67-8468dff1ea05, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-03-20T22:02:11.378880Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-03-20T22:01:47.831304Z, gen: Generation(Generation(3)), propolis_id: Some(b8a518ba-8675-4da7-b93b-394df66485d7), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(b8a518ba-8675-4da7-b93b-394df66485d7 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-03-20T22:02:11.645483565Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
27592025-03-20T22:02:11.650ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
27602025-03-20T22:02:11.652ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"d594712d-5385-423f-a318-6ea315a0e8a5","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"b8a518ba-8675-4da7-b93b-394df66485d7"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"d594712d-5385-423f-a318-6ea315a0e8a5","lookup_type":{"ById":"d594712d-5385-423f-a318-6ea315a0e8a5"},"parent":{"key":"e5716097-f669-4df9-b053-eb0ddb431a2d","lookup_type":{"ById":"e5716097-f669-4df9-b053-eb0ddb431a2d"},"parent":{"key":"4841a72c-7688-4028-b721-8d89119ea996","lookup_type":{"ById":"4841a72c-7688-4028-b721-8d89119ea996"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"59bb6c15-43c8-49e4-a9c6-a9681155d1d3"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"e5716097-f669-4df9-b053-eb0ddb431a2d","ram_diff":1073741824},"destroy_active_vmm":"b8a518ba-8675-4da7-b93b-394df66485d7","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-03-20T22:02:11.645483565Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27612025-03-20T22:02:11.652ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
27622025-03-20T22:02:11.657ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
27632025-03-20T22:02:11.657ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27642025-03-20T22:02:11.657ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27652025-03-20T22:02:11.657ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27662025-03-20T22:02:11.660ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27672025-03-20T22:02:11.660ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27682025-03-20T22:02:11.663ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27692025-03-20T22:02:11.663ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27702025-03-20T22:02:11.666ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27712025-03-20T22:02:11.666ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("8658e8b0-d354-431f-8e5a-42d447062057")) node_id = 0 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27722025-03-20T22:02:11.670ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27732025-03-20T22:02:11.670ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27742025-03-20T22:02:11.675ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 parent_lock = UpdaterLock { updater_id: 59bb6c15-43c8-49e4-a9c6-a9681155d1d3, locked_gen: Generation(Generation(2)) } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
27752025-03-20T22:02:11.675ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = BecomeUpdater
27762025-03-20T22:02:11.676ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = BecomeUpdater
27772025-03-20T22:02:11.690ZDEBGnexus (SagaExecutor): inherited lock from 59bb6c15-43c8-49e4-a9c6-a9681155d1d3 to 8658e8b0-d354-431f-8e5a-42d447062057 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 59bb6c15-43c8-49e4-a9c6-a9681155d1d3 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = BecomeUpdater updater_id = 8658e8b0-d354-431f-8e5a-42d447062057
27782025-03-20T22:02:11.690ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
27792025-03-20T22:02:11.690ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27802025-03-20T22:02:11.690ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("8658e8b0-d354-431f-8e5a-42d447062057")}) node_id = 1 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27812025-03-20T22:02:11.693ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27822025-03-20T22:02:11.693ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27832025-03-20T22:02:11.696ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27842025-03-20T22:02:11.696ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27852025-03-20T22:02:11.699ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
27862025-03-20T22:02:11.699ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
27872025-03-20T22:02:11.702ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
27882025-03-20T22:02:11.704ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5
27892025-03-20T22:02:11.704ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UpdateNetworkConfig
27902025-03-20T22:02:11.705ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UpdateNetworkConfig
27912025-03-20T22:02:11.709ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UpdateNetworkConfig
27922025-03-20T22:02:11.710ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UpdateNetworkConfig
27932025-03-20T22:02:11.712ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
27942025-03-20T22:02:11.712ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UpdateNetworkConfig
27952025-03-20T22:02:11.712ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UpdateNetworkConfig
27962025-03-20T22:02:11.714ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
27972025-03-20T22:02:11.714ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
27982025-03-20T22:02:11.714ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
27992025-03-20T22:02:11.716ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
28002025-03-20T22:02:11.717ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
28012025-03-20T22:02:11.717ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
28022025-03-20T22:02:11.717ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
28032025-03-20T22:02:11.724ZDEBGnexus: notifying dendrite of updates instance_id = Some(d594712d-5385-423f-a318-6ea315a0e8a5 (instance)) switch = switch0
28042025-03-20T22:02:11.725ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3675, tv_nsec: 531237657 } })
28052025-03-20T22:02:11.725ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1171
28062025-03-20T22:02:11.756ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
28072025-03-20T22:02:11.756ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28082025-03-20T22:02:11.756ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "55fae1eb-563b-4622-b75d-d1a0dd5f0de7", "content-length": "26", "date": "Thu, 20 Mar 2025 22:02:11 GMT"} })
28092025-03-20T22:02:11.756ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28102025-03-20T22:02:11.757ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28112025-03-20T22:02:11.786ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
28122025-03-20T22:02:11.786ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "09e6cf26-9bf2-45ef-8e54-a6b9c4f31aa4", "content-length": "4", "date": "Thu, 20 Mar 2025 22:02:11 GMT"} })
28132025-03-20T22:02:11.786ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28142025-03-20T22:02:11.786ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28152025-03-20T22:02:11.789ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28162025-03-20T22:02:11.789ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28172025-03-20T22:02:11.795ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
28182025-03-20T22:02:11.796ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
28192025-03-20T22:02:11.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 77b362a0-d4ec-49bc-bcf4-db5374c81077 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
28202025-03-20T22:02:11.804ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 77b362a0-d4ec-49bc-bcf4-db5374c81077 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
28212025-03-20T22:02:11.806ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2012 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 77b362a0-d4ec-49bc-bcf4-db5374c81077 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
28222025-03-20T22:02:11.809ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-03-20T22:02:11.798765Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 4841a72c-7688-4028-b721-8d89119ea996, time_modified: Some(2025-03-20T22:02:11.798765Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: e5716097-f669-4df9-b053-eb0ddb431a2d, time_modified: Some(2025-03-20T22:02:11.798765Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28232025-03-20T22:02:11.809ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28242025-03-20T22:02:11.809ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28252025-03-20T22:02:11.813ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28262025-03-20T22:02:11.813ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28272025-03-20T22:02:11.819ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28282025-03-20T22:02:11.819ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UnassignOximeterProducer
28292025-03-20T22:02:11.819ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UnassignOximeterProducer
28302025-03-20T22:02:11.823ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = c75d8631-e01d-47db-98de-f38076ccc33e producer_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28312025-03-20T22:02:11.823ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UnassignOximeterProducer
28322025-03-20T22:02:11.823ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = UnassignOximeterProducer
28332025-03-20T22:02:11.860ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28342025-03-20T22:02:11.860ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/d594712d-5385-423f-a318-6ea315a0e8a5
28352025-03-20T22:02:11.884ZDEBGnexus (SagaExecutor): client response oximeter-collector = d594712d-5385-423f-a318-6ea315a0e8a5 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/d594712d-5385-423f-a318-6ea315a0e8a5", status: 204, headers: {"x-request-id": "b082fda9-83ff-4950-92e2-1d898ed9dfda", "date": "Thu, 20 Mar 2025 22:02:11 GMT"} }) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28362025-03-20T22:02:11.884ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = c75d8631-e01d-47db-98de-f38076ccc33e producer_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28372025-03-20T22:02:11.885ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28382025-03-20T22:02:11.885ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28392025-03-20T22:02:11.889ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28402025-03-20T22:02:11.889ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28412025-03-20T22:02:11.892ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 lock = UpdaterLock { updater_id: 8658e8b0-d354-431f-8e5a-42d447062057, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-03-20T22:02:11.645483565Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28422025-03-20T22:02:11.892ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = CommitInstanceUpdates
28432025-03-20T22:02:11.892ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = CommitInstanceUpdates
28442025-03-20T22:02:11.963ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 new_runtime = InstanceRuntimeState { time_updated: 2025-03-20T22:02:11.645483565Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28452025-03-20T22:02:11.963ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28462025-03-20T22:02:11.963ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28472025-03-20T22:02:11.963ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Signaled
28482025-03-20T22:02:11.963ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
28492025-03-20T22:02:11.963ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Signaled
28502025-03-20T22:02:11.963ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28512025-03-20T22:02:11.963ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28522025-03-20T22:02:11.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28532025-03-20T22:02:11.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28542025-03-20T22:02:11.965ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28552025-03-20T22:02:11.967ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28562025-03-20T22:02:11.967ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28572025-03-20T22:02:11.967ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28582025-03-20T22:02:11.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28592025-03-20T22:02:11.967ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28602025-03-20T22:02:11.969ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28612025-03-20T22:02:11.971ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28622025-03-20T22:02:11.971ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28632025-03-20T22:02:11.971ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28642025-03-20T22:02:11.971ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d594712d-5385-423f-a318-6ea315a0e8a5"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("b8a518ba-8675-4da7-b93b-394df66485d7")}) node_id = 7 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28652025-03-20T22:02:11.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28662025-03-20T22:02:11.975ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28672025-03-20T22:02:11.975ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28682025-03-20T22:02:11.976ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28692025-03-20T22:02:11.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28702025-03-20T22:02:11.978ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28712025-03-20T22:02:11.978ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28722025-03-20T22:02:11.981ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28732025-03-20T22:02:11.981ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28742025-03-20T22:02:11.987ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28752025-03-20T22:02:11.987ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ReleaseSledResources
28762025-03-20T22:02:11.987ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ReleaseSledResources
28772025-03-20T22:02:11.996ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28782025-03-20T22:02:11.996ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28792025-03-20T22:02:11.999ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28802025-03-20T22:02:11.999ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28812025-03-20T22:02:12.002ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 propolis_id = b8a518ba-8675-4da7-b93b-394df66485d7 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
28822025-03-20T22:02:12.002ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = MarkVmmDeleted
28832025-03-20T22:02:12.003ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = MarkVmmDeleted
28842025-03-20T22:02:12.051ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28852025-03-20T22:02:12.051ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28862025-03-20T22:02:12.056ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28872025-03-20T22:02:12.056ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28882025-03-20T22:02:12.064ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
28892025-03-20T22:02:12.064ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28902025-03-20T22:02:12.065ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6b24689-da44-4dca-9672-3ce030af2131", "content-length": "372", "date": "Thu, 20 Mar 2025 22:02:12 GMT"} })
28912025-03-20T22:02:12.065ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28922025-03-20T22:02:12.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28932025-03-20T22:02:12.068ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28942025-03-20T22:02:12.068ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28952025-03-20T22:02:12.070ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28962025-03-20T22:02:12.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28972025-03-20T22:02:12.071ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
28982025-03-20T22:02:12.071ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
28992025-03-20T22:02:12.072ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29002025-03-20T22:02:12.074ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29012025-03-20T22:02:12.074ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29022025-03-20T22:02:12.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29032025-03-20T22:02:12.075ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29042025-03-20T22:02:12.075ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29052025-03-20T22:02:12.075ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29062025-03-20T22:02:12.077ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29072025-03-20T22:02:12.077ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29082025-03-20T22:02:12.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29092025-03-20T22:02:12.079ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
29102025-03-20T22:02:12.079ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
29112025-03-20T22:02:12.080ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c889d992-ce31-42da-95bd-f8331b9353ce", "date": "Thu, 20 Mar 2025 22:02:12 GMT"} })
29122025-03-20T22:02:12.080ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29132025-03-20T22:02:12.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29142025-03-20T22:02:12.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29152025-03-20T22:02:12.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29162025-03-20T22:02:12.085ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29172025-03-20T22:02:12.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29182025-03-20T22:02:12.092ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
29192025-03-20T22:02:12.092ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29202025-03-20T22:02:12.093ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "49641270-0325-49e4-83c5-f9d3515794dd", "date": "Thu, 20 Mar 2025 22:02:12 GMT"} })
29212025-03-20T22:02:12.094ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 130.683836ms iteration = 24 status = {}
29222025-03-20T22:02:12.109ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29232025-03-20T22:02:12.109ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29242025-03-20T22:02:12.110ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29252025-03-20T22:02:12.111ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29262025-03-20T22:02:12.112ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
29272025-03-20T22:02:12.112ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ae53913-0bf7-4c77-afe5-c563340001cc", "content-length": "691", "date": "Thu, 20 Mar 2025 22:02:12 GMT"} })
29282025-03-20T22:02:12.112ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
29292025-03-20T22:02:12.112ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
29302025-03-20T22:02:12.112ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
29312025-03-20T22:02:12.114ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 150.340992ms iteration = 24 status = {}
29322025-03-20T22:02:12.140ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29332025-03-20T22:02:12.144ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 4841a72c-7688-4028-b721-8d89119ea996, lookup_type: ById(4841a72c-7688-4028-b721-8d89119ea996) }, key: e5716097-f669-4df9-b053-eb0ddb431a2d, lookup_type: ById(e5716097-f669-4df9-b053-eb0ddb431a2d) }, key: d594712d-5385-423f-a318-6ea315a0e8a5, lookup_type: ById(d594712d-5385-423f-a318-6ea315a0e8a5) } result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29342025-03-20T22:02:12.144ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29352025-03-20T22:02:12.144ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_node = ChainSuccessorSaga
29362025-03-20T22:02:12.154ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 karmic_state = WillReincarnate needs_reincarnation = false saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update
29372025-03-20T22:02:12.154ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29382025-03-20T22:02:12.154ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
29392025-03-20T22:02:12.158ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29402025-03-20T22:02:12.158ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
29412025-03-20T22:02:12.162ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29422025-03-20T22:02:12.162ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
29432025-03-20T22:02:12.166ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29442025-03-20T22:02:12.166ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 13373b56-8236-44fe-82a6-44c47f425d81
29452025-03-20T22:02:12.179ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29462025-03-20T22:02:12.179ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 saga_name = instance-update saga_result = SagaResult { saga_id: 13373b56-8236-44fe-82a6-44c47f425d81, saga_log: SagaLog { saga_id: 13373b56-8236-44fe-82a6-44c47f425d81, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("8658e8b0-d354-431f-8e5a-42d447062057")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("8658e8b0-d354-431f-8e5a-42d447062057")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("d594712d-5385-423f-a318-6ea315a0e8a5"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("b8a518ba-8675-4da7-b93b-394df66485d7")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("d594712d-5385-423f-a318-6ea315a0e8a5"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("b8a518ba-8675-4da7-b93b-394df66485d7")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("8658e8b0-d354-431f-8e5a-42d447062057"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("8658e8b0-d354-431f-8e5a-42d447062057")}} }) }
29472025-03-20T22:02:12.179ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 13373b56-8236-44fe-82a6-44c47f425d81 instance_id = d594712d-5385-423f-a318-6ea315a0e8a5 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update
29482025-03-20T22:02:12.179ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29492025-03-20T22:02:12.179ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
29502025-03-20T22:02:12.183ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29512025-03-20T22:02:12.183ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
29522025-03-20T22:02:12.186ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29532025-03-20T22:02:12.186ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
29542025-03-20T22:02:12.190ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29552025-03-20T22:02:12.190ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a
29562025-03-20T22:02:12.223ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update sec_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
29572025-03-20T22:02:12.223ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = b6503eba-9bdb-4cdb-bbad-51d9833ba73a saga_name = start-instance-update saga_result = SagaResult { saga_id: b6503eba-9bdb-4cdb-bbad-51d9833ba73a, saga_log: SagaLog { saga_id: b6503eba-9bdb-4cdb-bbad-51d9833ba73a, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("59bb6c15-43c8-49e4-a9c6-a9681155d1d3")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("59bb6c15-43c8-49e4-a9c6-a9681155d1d3")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("59bb6c15-43c8-49e4-a9c6-a9681155d1d3"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("59bb6c15-43c8-49e4-a9c6-a9681155d1d3")}} }) }
29582025-03-20T22:02:12.827ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8fa1df08-c589-40fd-b786-e870bad1add2 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29592025-03-20T22:02:12.827ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8fa1df08-c589-40fd-b786-e870bad1add2 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29602025-03-20T22:02:12.829ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2443 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8fa1df08-c589-40fd-b786-e870bad1add2 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29612025-03-20T22:02:13.850ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e930aca9-c2f3-414c-b4a3-1dd71065fe68 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29622025-03-20T22:02:13.850ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e930aca9-c2f3-414c-b4a3-1dd71065fe68 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29632025-03-20T22:02:13.852ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2179 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e930aca9-c2f3-414c-b4a3-1dd71065fe68 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29642025-03-20T22:02:14.873ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 170824da-5404-4fd0-9607-abf365421bff roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29652025-03-20T22:02:14.873ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 170824da-5404-4fd0-9607-abf365421bff resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29662025-03-20T22:02:14.875ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1808 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 170824da-5404-4fd0-9607-abf365421bff response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29672025-03-20T22:02:15.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
29682025-03-20T22:02:15.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29692025-03-20T22:02:15.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
29702025-03-20T22:02:15.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29712025-03-20T22:02:15.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29722025-03-20T22:02:15.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29732025-03-20T22:02:15.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29742025-03-20T22:02:15.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29752025-03-20T22:02:15.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29762025-03-20T22:02:15.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29772025-03-20T22:02:15.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29782025-03-20T22:02:15.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29792025-03-20T22:02:15.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29802025-03-20T22:02:15.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
29812025-03-20T22:02:15.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29822025-03-20T22:02:15.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29832025-03-20T22:02:15.258ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
29842025-03-20T22:02:15.258ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.982115ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
29852025-03-20T22:02:15.268ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29862025-03-20T22:02:15.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29872025-03-20T22:02:15.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.541712ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29882025-03-20T22:02:15.895ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b8400094-cd46-47f9-b66b-8616358a8370 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29892025-03-20T22:02:15.895ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b8400094-cd46-47f9-b66b-8616358a8370 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29902025-03-20T22:02:15.897ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2136 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b8400094-cd46-47f9-b66b-8616358a8370 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29912025-03-20T22:02:16.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d047f39-c6ec-4618-84be-2f44fa17f4c5 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29922025-03-20T22:02:16.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d047f39-c6ec-4618-84be-2f44fa17f4c5 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29932025-03-20T22:02:16.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2315 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3d047f39-c6ec-4618-84be-2f44fa17f4c5 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
29942025-03-20T22:02:17.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 20749bfc-0ee8-4349-91c1-bd735eb948da roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
29952025-03-20T22:02:17.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 20749bfc-0ee8-4349-91c1-bd735eb948da resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
29962025-03-20T22:02:17.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2534 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 20749bfc-0ee8-4349-91c1-bd735eb948da response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
29972025-03-20T22:02:17.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 5f10525a-feda-4eb3-9b56-df2da2bd337f roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
29982025-03-20T22:02:17.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 5f10525a-feda-4eb3-9b56-df2da2bd337f resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
29992025-03-20T22:02:17.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2640 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 5f10525a-feda-4eb3-9b56-df2da2bd337f response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
30002025-03-20T22:02:17.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 605 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = bca24f75-eeee-4e2f-abec-b3588f228e3c response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
30012025-03-20T22:02:17.940ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5302769c-64a4-4c62-9f56-a6cfe6e54f0c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30022025-03-20T22:02:17.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5302769c-64a4-4c62-9f56-a6cfe6e54f0c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30032025-03-20T22:02:17.943ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2530 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5302769c-64a4-4c62-9f56-a6cfe6e54f0c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30042025-03-20T22:02:18.963ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 95d9159c-b88e-48e9-aaec-892ae67430aa roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30052025-03-20T22:02:18.963ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 95d9159c-b88e-48e9-aaec-892ae67430aa resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30062025-03-20T22:02:18.965ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2558 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 95d9159c-b88e-48e9-aaec-892ae67430aa response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30072025-03-20T22:02:19.984ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8c23d82-8bb1-4a22-a5c7-8548a66c5ebb roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30082025-03-20T22:02:19.985ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8c23d82-8bb1-4a22-a5c7-8548a66c5ebb resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30092025-03-20T22:02:19.987ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2485 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8c23d82-8bb1-4a22-a5c7-8548a66c5ebb response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30102025-03-20T22:02:21.008ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 73562250-199f-4100-86eb-385c22f7222c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30112025-03-20T22:02:21.008ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 73562250-199f-4100-86eb-385c22f7222c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30122025-03-20T22:02:21.010ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2752 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 73562250-199f-4100-86eb-385c22f7222c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30132025-03-20T22:02:22.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80cdce97-1a95-461c-aa2f-e52261d7c326 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30142025-03-20T22:02:22.032ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80cdce97-1a95-461c-aa2f-e52261d7c326 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30152025-03-20T22:02:22.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2402 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80cdce97-1a95-461c-aa2f-e52261d7c326 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30162025-03-20T22:02:23.053ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1adbbd66-86b6-467a-b73f-e9570a006877 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30172025-03-20T22:02:23.054ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1adbbd66-86b6-467a-b73f-e9570a006877 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30182025-03-20T22:02:23.056ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2945 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1adbbd66-86b6-467a-b73f-e9570a006877 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30192025-03-20T22:02:24.076ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0dbcd780-0998-42df-b267-93322531cb06 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30202025-03-20T22:02:24.076ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0dbcd780-0998-42df-b267-93322531cb06 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30212025-03-20T22:02:24.079ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2890 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0dbcd780-0998-42df-b267-93322531cb06 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30222025-03-20T22:02:25.097ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c5d8a40a-f52a-46f3-95e1-7e29b080c326 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30232025-03-20T22:02:25.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c5d8a40a-f52a-46f3-95e1-7e29b080c326 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30242025-03-20T22:02:25.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2748 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c5d8a40a-f52a-46f3-95e1-7e29b080c326 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30252025-03-20T22:02:25.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
30262025-03-20T22:02:25.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30272025-03-20T22:02:25.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
30282025-03-20T22:02:25.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30292025-03-20T22:02:25.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30302025-03-20T22:02:25.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30312025-03-20T22:02:25.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30322025-03-20T22:02:25.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30332025-03-20T22:02:25.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30342025-03-20T22:02:25.242ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.822403ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30352025-03-20T22:02:25.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30362025-03-20T22:02:25.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30372025-03-20T22:02:25.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30382025-03-20T22:02:25.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30392025-03-20T22:02:25.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30402025-03-20T22:02:25.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30412025-03-20T22:02:25.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
30422025-03-20T22:02:25.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30432025-03-20T22:02:25.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30442025-03-20T22:02:25.264ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
30452025-03-20T22:02:25.264ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.579044ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
30462025-03-20T22:02:26.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cb1f3a63-fd6b-46b4-92bf-48462e6ec2ac roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30472025-03-20T22:02:26.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cb1f3a63-fd6b-46b4-92bf-48462e6ec2ac resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30482025-03-20T22:02:26.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1894 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cb1f3a63-fd6b-46b4-92bf-48462e6ec2ac response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30492025-03-20T22:02:27.141ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 47d49940-9b80-49c3-aa99-d6270babdfd6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30502025-03-20T22:02:27.142ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 47d49940-9b80-49c3-aa99-d6270babdfd6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30512025-03-20T22:02:27.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2793 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 47d49940-9b80-49c3-aa99-d6270babdfd6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30522025-03-20T22:02:27.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 719 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 3cf10f83-9a35-458b-9d0f-c75283f4ec21 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
30532025-03-20T22:02:28.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2fc6b57a-1a6d-4ab2-ac63-1384a7859859 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30542025-03-20T22:02:28.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2fc6b57a-1a6d-4ab2-ac63-1384a7859859 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30552025-03-20T22:02:28.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3139 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2fc6b57a-1a6d-4ab2-ac63-1384a7859859 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30562025-03-20T22:02:29.189ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0c4e4571-ac6b-487a-9436-c41b5190c76a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30572025-03-20T22:02:29.189ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0c4e4571-ac6b-487a-9436-c41b5190c76a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30582025-03-20T22:02:29.191ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2417 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0c4e4571-ac6b-487a-9436-c41b5190c76a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30592025-03-20T22:02:30.210ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c5cb013-9f3d-4207-bb40-7b3c70873a11 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30602025-03-20T22:02:30.210ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c5cb013-9f3d-4207-bb40-7b3c70873a11 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30612025-03-20T22:02:30.212ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2095 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c5cb013-9f3d-4207-bb40-7b3c70873a11 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30622025-03-20T22:02:31.231ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 846bcd74-93e5-4cf4-87cf-5aadee467163 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30632025-03-20T22:02:31.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 846bcd74-93e5-4cf4-87cf-5aadee467163 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30642025-03-20T22:02:31.233ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1954 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 846bcd74-93e5-4cf4-87cf-5aadee467163 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30652025-03-20T22:02:32.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 97e7e966-6b75-4a6c-8ee7-1d0d040cf484 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
30662025-03-20T22:02:32.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 97e7e966-6b75-4a6c-8ee7-1d0d040cf484 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
30672025-03-20T22:02:32.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3058 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 97e7e966-6b75-4a6c-8ee7-1d0d040cf484 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
30682025-03-20T22:02:32.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 4372ae64-4a77-446e-9662-5cd73cba8c12 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
30692025-03-20T22:02:32.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 4372ae64-4a77-446e-9662-5cd73cba8c12 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
30702025-03-20T22:02:32.253ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9eb0d75e-b897-418b-88b3-ee8d70f9ecd4 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30712025-03-20T22:02:32.253ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9eb0d75e-b897-418b-88b3-ee8d70f9ecd4 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30722025-03-20T22:02:32.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2593 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 4372ae64-4a77-446e-9662-5cd73cba8c12 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
30732025-03-20T22:02:32.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2301 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9eb0d75e-b897-418b-88b3-ee8d70f9ecd4 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30742025-03-20T22:02:33.274ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9b12792a-0aac-48ec-85f3-12a3c35d7bc4 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30752025-03-20T22:02:33.275ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9b12792a-0aac-48ec-85f3-12a3c35d7bc4 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30762025-03-20T22:02:33.277ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2577 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9b12792a-0aac-48ec-85f3-12a3c35d7bc4 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30772025-03-20T22:02:34.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a28c5888-979f-44c7-aa7a-1f705be5b8ed roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30782025-03-20T22:02:34.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a28c5888-979f-44c7-aa7a-1f705be5b8ed resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30792025-03-20T22:02:34.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2367 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a28c5888-979f-44c7-aa7a-1f705be5b8ed response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
30802025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
30812025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
30822025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
30832025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30842025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
30852025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
30862025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30872025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
30882025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
30892025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
30902025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30912025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
30922025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
30932025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
30942025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30952025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30962025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
30972025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30982025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
30992025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31002025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
31012025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
31022025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
31032025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
31042025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
31052025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31062025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
31072025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
31082025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31092025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
31102025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31112025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31122025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
31132025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31142025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
31152025-03-20T22:02:35.238ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
31162025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31172025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
31182025-03-20T22:02:35.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31192025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
31202025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31212025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31222025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
31232025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31242025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31252025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31262025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31272025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31282025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
31292025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31302025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31312025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31322025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31332025-03-20T22:02:35.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31342025-03-20T22:02:35.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
31352025-03-20T22:02:35.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
31362025-03-20T22:02:35.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31372025-03-20T22:02:35.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31382025-03-20T22:02:35.245ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.829742ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
31392025-03-20T22:02:35.245ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31402025-03-20T22:02:35.246ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3699, tv_nsec: 52523991 } })
31412025-03-20T22:02:35.246ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
31422025-03-20T22:02:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31432025-03-20T22:02:35.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31442025-03-20T22:02:35.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31452025-03-20T22:02:35.247ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
31462025-03-20T22:02:35.247ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31472025-03-20T22:02:35.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31482025-03-20T22:02:35.251ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31492025-03-20T22:02:35.252ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31502025-03-20T22:02:35.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31512025-03-20T22:02:35.254ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31522025-03-20T22:02:35.254ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31532025-03-20T22:02:35.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31542025-03-20T22:02:35.255ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31552025-03-20T22:02:35.256ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3699, tv_nsec: 62199830 } })
31562025-03-20T22:02:35.256ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
31572025-03-20T22:02:35.257ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31582025-03-20T22:02:35.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31592025-03-20T22:02:35.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31602025-03-20T22:02:35.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
31612025-03-20T22:02:35.260ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31622025-03-20T22:02:35.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31632025-03-20T22:02:35.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
31642025-03-20T22:02:35.260ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31652025-03-20T22:02:35.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31662025-03-20T22:02:35.266ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 27.335431ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31672025-03-20T22:02:35.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31682025-03-20T22:02:35.267ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31692025-03-20T22:02:35.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31702025-03-20T22:02:35.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
31712025-03-20T22:02:35.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31722025-03-20T22:02:35.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31732025-03-20T22:02:35.288ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
31742025-03-20T22:02:35.288ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.197512ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
31752025-03-20T22:02:35.303ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31762025-03-20T22:02:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
31772025-03-20T22:02:35.306ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31782025-03-20T22:02:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31792025-03-20T22:02:35.307ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
31802025-03-20T22:02:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
31812025-03-20T22:02:35.309ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3699, tv_nsec: 115829524 } })
31822025-03-20T22:02:35.310ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
31832025-03-20T22:02:35.311ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31842025-03-20T22:02:35.311ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 72.853199ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
31852025-03-20T22:02:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
31862025-03-20T22:02:35.312ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 74.266557ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
31872025-03-20T22:02:35.312ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31882025-03-20T22:02:35.313ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
31892025-03-20T22:02:35.313ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31902025-03-20T22:02:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31912025-03-20T22:02:35.314ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922025-03-20T22:02:35.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31932025-03-20T22:02:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31942025-03-20T22:02:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31952025-03-20T22:02:35.316ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31962025-03-20T22:02:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31972025-03-20T22:02:35.316ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31982025-03-20T22:02:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31992025-03-20T22:02:35.317ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 79.24196ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
32002025-03-20T22:02:35.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 93d7ac70-3b52-43a6-aa3a-35f41c9a1e82 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
32012025-03-20T22:02:35.319ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 93d7ac70-3b52-43a6-aa3a-35f41c9a1e82 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
32022025-03-20T22:02:35.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32032025-03-20T22:02:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32042025-03-20T22:02:35.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32052025-03-20T22:02:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32062025-03-20T22:02:35.321ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32072025-03-20T22:02:35.321ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2335 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 93d7ac70-3b52-43a6-aa3a-35f41c9a1e82 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
32082025-03-20T22:02:35.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32092025-03-20T22:02:35.322ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32102025-03-20T22:02:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32112025-03-20T22:02:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32122025-03-20T22:02:35.323ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32132025-03-20T22:02:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32142025-03-20T22:02:35.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32152025-03-20T22:02:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
32162025-03-20T22:02:35.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32172025-03-20T22:02:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32182025-03-20T22:02:35.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32192025-03-20T22:02:35.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32202025-03-20T22:02:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32212025-03-20T22:02:35.331ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
32222025-03-20T22:02:35.331ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 93.130611ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
32232025-03-20T22:02:35.334ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
32242025-03-20T22:02:35.334ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32252025-03-20T22:02:35.334ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "837e62db-978a-4b0d-8d19-e762ea30b056", "content-length": "26", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32262025-03-20T22:02:35.334ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32272025-03-20T22:02:35.335ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.43671ms iteration = 24 status = 0
32282025-03-20T22:02:35.339ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32292025-03-20T22:02:35.348ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
32302025-03-20T22:02:35.348ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32312025-03-20T22:02:35.348ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f685915-f4cd-4418-9bd9-094ac34ce771", "content-length": "26", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32322025-03-20T22:02:35.348ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32332025-03-20T22:02:35.350ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32342025-03-20T22:02:35.354ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.765856ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
32352025-03-20T22:02:35.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 119.782353ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
32362025-03-20T22:02:35.359ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32372025-03-20T22:02:35.359ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
32382025-03-20T22:02:35.359ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32392025-03-20T22:02:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32402025-03-20T22:02:35.364ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32412025-03-20T22:02:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32422025-03-20T22:02:35.376ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32432025-03-20T22:02:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32442025-03-20T22:02:35.380ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32452025-03-20T22:02:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32462025-03-20T22:02:35.387ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
32472025-03-20T22:02:35.387ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32482025-03-20T22:02:35.387ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c05deae-8203-401c-8b94-ca1d545a5493", "content-length": "26", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32492025-03-20T22:02:35.387ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32502025-03-20T22:02:35.390ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
32512025-03-20T22:02:35.393ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
32522025-03-20T22:02:35.393ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
32532025-03-20T22:02:35.393ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 154.820991ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
32542025-03-20T22:02:35.399ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 160.691838ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
32552025-03-20T22:02:35.402ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32562025-03-20T22:02:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32572025-03-20T22:02:35.410ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32582025-03-20T22:02:35.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32592025-03-20T22:02:35.411ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32602025-03-20T22:02:35.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32612025-03-20T22:02:35.413ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32622025-03-20T22:02:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32632025-03-20T22:02:35.418ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
32642025-03-20T22:02:35.418ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 179.742768ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
32652025-03-20T22:02:35.434ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 196.084985ms iteration = 27 status = {"physical_disks_added":0}
32662025-03-20T22:02:35.449ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 210.652905ms iteration = 24 status = {}
32672025-03-20T22:02:35.475ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
32682025-03-20T22:02:35.476ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6e30612-0a39-48c5-834f-75a6d1c58f5e", "content-length": "2", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32692025-03-20T22:02:35.477ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32702025-03-20T22:02:35.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32712025-03-20T22:02:35.519ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
32722025-03-20T22:02:35.519ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32732025-03-20T22:02:35.519ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "10d27795-429d-4c93-8faa-17c19cb69469", "content-length": "372", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32742025-03-20T22:02:35.520ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32752025-03-20T22:02:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32762025-03-20T22:02:35.521ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32772025-03-20T22:02:35.523ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 284.034257ms iteration = 24 status = null
32782025-03-20T22:02:35.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32792025-03-20T22:02:35.526ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32802025-03-20T22:02:35.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32812025-03-20T22:02:35.538ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
32822025-03-20T22:02:35.538ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
32832025-03-20T22:02:35.538ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "15042f9b-50eb-46d2-b0cb-74fd9589bc1c", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32842025-03-20T22:02:35.538ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32852025-03-20T22:02:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32862025-03-20T22:02:35.543ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32872025-03-20T22:02:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32882025-03-20T22:02:35.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32892025-03-20T22:02:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32902025-03-20T22:02:35.555ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
32912025-03-20T22:02:35.555ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32922025-03-20T22:02:35.556ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "12c7dff1-2721-455a-b40b-bcc645208d33", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32932025-03-20T22:02:35.558ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 319.27115ms iteration = 25 status = {}
32942025-03-20T22:02:35.610ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32952025-03-20T22:02:35.610ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb8b820d-4551-46f3-88aa-f06e7400ea55", "content-length": "691", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
32962025-03-20T22:02:35.610ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
32972025-03-20T22:02:35.610ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
32982025-03-20T22:02:35.610ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
32992025-03-20T22:02:35.611ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 373.068918ms iteration = 25 status = {}
33002025-03-20T22:02:35.614ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33012025-03-20T22:02:35.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33022025-03-20T22:02:35.616ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
33032025-03-20T22:02:35.616ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33042025-03-20T22:02:35.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33052025-03-20T22:02:35.626ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
33062025-03-20T22:02:35.627ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d0668e1-44b4-43aa-8ff0-b1a52543272f", "content-length": "134", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
33072025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
33082025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
33092025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
33102025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
33112025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33122025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33132025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33142025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33152025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33162025-03-20T22:02:35.627ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33172025-03-20T22:02:35.628ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
33182025-03-20T22:02:35.628ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33192025-03-20T22:02:35.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33202025-03-20T22:02:35.630ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
33212025-03-20T22:02:35.630ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa9beba3-561a-47bb-b3e8-90a7e2b40750", "content-length": "2", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
33222025-03-20T22:02:35.630ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
33232025-03-20T22:02:35.630ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0207ff6-11f0-450e-8f6d-c326a860ca1b", "content-length": "52", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
33242025-03-20T22:02:35.630ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
33252025-03-20T22:02:35.630ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
33262025-03-20T22:02:35.730ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "60563035-ca8e-4b75-87e4-10ee69537e08", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
33272025-03-20T22:02:35.730ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
33282025-03-20T22:02:35.730ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
33292025-03-20T22:02:35.731ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a4a4d11-8e3b-40ac-b089-9812653eecd6", "content-length": "682", "date": "Thu, 20 Mar 2025 22:02:35 GMT"} })
33302025-03-20T22:02:35.731ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33312025-03-20T22:02:35.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33322025-03-20T22:02:35.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33332025-03-20T22:02:35.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33342025-03-20T22:02:35.738ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33352025-03-20T22:02:35.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33362025-03-20T22:02:35.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33372025-03-20T22:02:35.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33382025-03-20T22:02:35.747ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 509.093427ms iteration = 24 status = {}
33392025-03-20T22:02:36.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 900057cd-d6b2-4ee7-87bc-19c260491381 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33402025-03-20T22:02:36.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 900057cd-d6b2-4ee7-87bc-19c260491381 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33412025-03-20T22:02:36.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2502 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 900057cd-d6b2-4ee7-87bc-19c260491381 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33422025-03-20T22:02:37.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 987 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = f5a12fc3-00a3-4ff1-ba67-ca8a1d0752bc response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
33432025-03-20T22:02:37.369ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0df6d98-414d-43b1-bb42-9f5bbf8a2c97 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33442025-03-20T22:02:37.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0df6d98-414d-43b1-bb42-9f5bbf8a2c97 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33452025-03-20T22:02:37.371ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2347 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0df6d98-414d-43b1-bb42-9f5bbf8a2c97 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33462025-03-20T22:02:38.392ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80ac7352-6b60-439c-9860-6f0b3c0d9076 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33472025-03-20T22:02:38.392ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80ac7352-6b60-439c-9860-6f0b3c0d9076 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33482025-03-20T22:02:38.394ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2606 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80ac7352-6b60-439c-9860-6f0b3c0d9076 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33492025-03-20T22:02:39.414ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4da282f2-6dc5-4c4d-8c75-7cfef7f4664d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33502025-03-20T22:02:39.414ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4da282f2-6dc5-4c4d-8c75-7cfef7f4664d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33512025-03-20T22:02:39.416ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2693 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4da282f2-6dc5-4c4d-8c75-7cfef7f4664d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33522025-03-20T22:02:40.435ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ba361c8a-5c00-4510-bb69-12044a8498ff roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33532025-03-20T22:02:40.436ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ba361c8a-5c00-4510-bb69-12044a8498ff resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33542025-03-20T22:02:40.438ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2883 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ba361c8a-5c00-4510-bb69-12044a8498ff response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33552025-03-20T22:02:41.457ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 13e6f8b6-bc26-4e14-b975-b1655e944642 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33562025-03-20T22:02:41.458ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 13e6f8b6-bc26-4e14-b975-b1655e944642 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33572025-03-20T22:02:41.460ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3065 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 13e6f8b6-bc26-4e14-b975-b1655e944642 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33582025-03-20T22:02:42.479ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8b19fe2-aa0c-44c7-bdef-e31928ddca79 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33592025-03-20T22:02:42.480ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8b19fe2-aa0c-44c7-bdef-e31928ddca79 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33602025-03-20T22:02:42.482ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2249 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d8b19fe2-aa0c-44c7-bdef-e31928ddca79 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33612025-03-20T22:02:43.501ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b4b3e179-2c39-4a05-ac6f-d76996a0b34f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33622025-03-20T22:02:43.501ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b4b3e179-2c39-4a05-ac6f-d76996a0b34f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33632025-03-20T22:02:43.503ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2594 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b4b3e179-2c39-4a05-ac6f-d76996a0b34f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33642025-03-20T22:02:44.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e6e76d1a-a0e4-4e5e-8ff1-7546741e37b9 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33652025-03-20T22:02:44.523ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e6e76d1a-a0e4-4e5e-8ff1-7546741e37b9 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33662025-03-20T22:02:44.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2344 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e6e76d1a-a0e4-4e5e-8ff1-7546741e37b9 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33672025-03-20T22:02:45.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
33682025-03-20T22:02:45.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
33692025-03-20T22:02:45.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33702025-03-20T22:02:45.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33712025-03-20T22:02:45.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33722025-03-20T22:02:45.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33732025-03-20T22:02:45.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33742025-03-20T22:02:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33752025-03-20T22:02:45.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33762025-03-20T22:02:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33772025-03-20T22:02:45.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33782025-03-20T22:02:45.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33792025-03-20T22:02:45.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33802025-03-20T22:02:45.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
33812025-03-20T22:02:45.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33822025-03-20T22:02:45.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33832025-03-20T22:02:45.256ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
33842025-03-20T22:02:45.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.10282ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
33852025-03-20T22:02:45.268ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33862025-03-20T22:02:45.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33872025-03-20T22:02:45.269ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.210012ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33882025-03-20T22:02:45.545ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f6eb9e24-0011-4b4d-b635-9e725011a965 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33892025-03-20T22:02:45.545ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f6eb9e24-0011-4b4d-b635-9e725011a965 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33902025-03-20T22:02:45.547ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2059 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f6eb9e24-0011-4b4d-b635-9e725011a965 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33912025-03-20T22:02:46.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ac4f25f1-7a17-43f7-9a29-8396a9d384d8 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33922025-03-20T22:02:46.568ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ac4f25f1-7a17-43f7-9a29-8396a9d384d8 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33932025-03-20T22:02:46.570ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2611 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ac4f25f1-7a17-43f7-9a29-8396a9d384d8 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33942025-03-20T22:02:47.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 557 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 849f72ac-72d9-41df-97ca-3484bc979056 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
33952025-03-20T22:02:47.591ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0992fb5-ad83-4cfc-a3d6-17ff9ff7ea0e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33962025-03-20T22:02:47.591ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0992fb5-ad83-4cfc-a3d6-17ff9ff7ea0e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33972025-03-20T22:02:47.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2814 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0992fb5-ad83-4cfc-a3d6-17ff9ff7ea0e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33982025-03-20T22:02:48.613ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a0085ee-c476-472f-b9a5-32f2879bdce9 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
33992025-03-20T22:02:48.614ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a0085ee-c476-472f-b9a5-32f2879bdce9 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34002025-03-20T22:02:48.615ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2233 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a0085ee-c476-472f-b9a5-32f2879bdce9 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34012025-03-20T22:02:49.636ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1f566f80-de03-4ce5-9b22-73bf8c15ee20 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34022025-03-20T22:02:49.636ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1f566f80-de03-4ce5-9b22-73bf8c15ee20 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34032025-03-20T22:02:49.638ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2416 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1f566f80-de03-4ce5-9b22-73bf8c15ee20 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34042025-03-20T22:02:50.656ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a31f84f-bd58-48cf-95ea-ebf1b40184fd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34052025-03-20T22:02:50.657ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a31f84f-bd58-48cf-95ea-ebf1b40184fd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34062025-03-20T22:02:50.659ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2878 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3a31f84f-bd58-48cf-95ea-ebf1b40184fd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34072025-03-20T22:02:51.678ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 63b4673e-b09a-4032-b1c6-263d8bdd975c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34082025-03-20T22:02:51.679ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 63b4673e-b09a-4032-b1c6-263d8bdd975c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34092025-03-20T22:02:51.681ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2292 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 63b4673e-b09a-4032-b1c6-263d8bdd975c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34102025-03-20T22:02:52.700ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3faf8369-d3d5-42cd-a8ea-a6d093a9b60a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34112025-03-20T22:02:52.701ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3faf8369-d3d5-42cd-a8ea-a6d093a9b60a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34122025-03-20T22:02:52.703ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2481 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3faf8369-d3d5-42cd-a8ea-a6d093a9b60a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34132025-03-20T22:02:53.723ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b515c83a-801a-4916-83ae-1942c0dece90 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34142025-03-20T22:02:53.724ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b515c83a-801a-4916-83ae-1942c0dece90 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34152025-03-20T22:02:53.725ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2237 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b515c83a-801a-4916-83ae-1942c0dece90 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34162025-03-20T22:02:54.744ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1d0f7e0d-c0b5-4ac6-bee9-d78b63fdfe09 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34172025-03-20T22:02:54.745ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1d0f7e0d-c0b5-4ac6-bee9-d78b63fdfe09 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34182025-03-20T22:02:54.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2456 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1d0f7e0d-c0b5-4ac6-bee9-d78b63fdfe09 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34192025-03-20T22:02:55.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
34202025-03-20T22:02:55.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
34212025-03-20T22:02:55.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34222025-03-20T22:02:55.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34232025-03-20T22:02:55.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34242025-03-20T22:02:55.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34252025-03-20T22:02:55.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34262025-03-20T22:02:55.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34272025-03-20T22:02:55.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34282025-03-20T22:02:55.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34292025-03-20T22:02:55.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34302025-03-20T22:02:55.242ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.383877ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34312025-03-20T22:02:55.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34322025-03-20T22:02:55.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34332025-03-20T22:02:55.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34342025-03-20T22:02:55.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34352025-03-20T22:02:55.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
34362025-03-20T22:02:55.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34372025-03-20T22:02:55.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34382025-03-20T22:02:55.256ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
34392025-03-20T22:02:55.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.656099ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
34402025-03-20T22:02:55.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22664ed2-e3c2-4cf6-baf3-9ad65a764710 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34412025-03-20T22:02:55.767ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22664ed2-e3c2-4cf6-baf3-9ad65a764710 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34422025-03-20T22:02:55.769ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2010 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22664ed2-e3c2-4cf6-baf3-9ad65a764710 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34432025-03-20T22:02:56.788ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 71227b04-2fc2-4ea7-918a-ad480409d975 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34442025-03-20T22:02:56.788ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 71227b04-2fc2-4ea7-918a-ad480409d975 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34452025-03-20T22:02:56.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2377 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 71227b04-2fc2-4ea7-918a-ad480409d975 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34462025-03-20T22:02:57.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 667 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = eb356798-6f63-40cd-aa5c-d5260d7cfc37 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
34472025-03-20T22:02:57.809ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2e0c1094-5992-4233-83b0-394ce190f0b9 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34482025-03-20T22:02:57.810ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2e0c1094-5992-4233-83b0-394ce190f0b9 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34492025-03-20T22:02:57.811ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2385 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2e0c1094-5992-4233-83b0-394ce190f0b9 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34502025-03-20T22:02:58.831ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f37b4e0b-91c5-4a4f-8785-77c305833e96 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34512025-03-20T22:02:58.831ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f37b4e0b-91c5-4a4f-8785-77c305833e96 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34522025-03-20T22:02:58.833ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2479 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f37b4e0b-91c5-4a4f-8785-77c305833e96 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34532025-03-20T22:02:59.853ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 01ee270b-dbe7-42e7-9d3b-2ca2681bb2aa roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34542025-03-20T22:02:59.853ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 01ee270b-dbe7-42e7-9d3b-2ca2681bb2aa resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34552025-03-20T22:02:59.855ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2442 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 01ee270b-dbe7-42e7-9d3b-2ca2681bb2aa response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34562025-03-20T22:03:00.874ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 37260248-c417-4408-9319-d864e6d7e048 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34572025-03-20T22:03:00.874ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 37260248-c417-4408-9319-d864e6d7e048 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34582025-03-20T22:03:00.877ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2544 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 37260248-c417-4408-9319-d864e6d7e048 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34592025-03-20T22:03:01.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53e44364-45bf-408a-a0f3-1d40be9bcd62 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34602025-03-20T22:03:01.896ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53e44364-45bf-408a-a0f3-1d40be9bcd62 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34612025-03-20T22:03:01.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2432 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 53e44364-45bf-408a-a0f3-1d40be9bcd62 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34622025-03-20T22:03:02.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 18adb1b8-4286-43de-b33e-37092c99637c roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
34632025-03-20T22:03:02.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 18adb1b8-4286-43de-b33e-37092c99637c resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
34642025-03-20T22:03:02.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3823 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 18adb1b8-4286-43de-b33e-37092c99637c response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
34652025-03-20T22:03:02.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 501c9888-e18c-48d1-9fec-dff89108e8e5 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
34662025-03-20T22:03:02.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 501c9888-e18c-48d1-9fec-dff89108e8e5 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
34672025-03-20T22:03:02.254ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2656 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 501c9888-e18c-48d1-9fec-dff89108e8e5 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
34682025-03-20T22:03:02.918ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 14fe2e1a-863b-49dc-96ae-73485489cd39 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34692025-03-20T22:03:02.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 14fe2e1a-863b-49dc-96ae-73485489cd39 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34702025-03-20T22:03:02.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2419 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 14fe2e1a-863b-49dc-96ae-73485489cd39 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34712025-03-20T22:03:03.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62ac0c1d-8bed-4050-8e0f-1df7da186a0e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34722025-03-20T22:03:03.941ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62ac0c1d-8bed-4050-8e0f-1df7da186a0e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34732025-03-20T22:03:03.943ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2426 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62ac0c1d-8bed-4050-8e0f-1df7da186a0e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34742025-03-20T22:03:04.961ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3401f182-6425-4c7b-9e7b-a7c99c39ee89 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34752025-03-20T22:03:04.961ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3401f182-6425-4c7b-9e7b-a7c99c39ee89 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34762025-03-20T22:03:04.963ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2360 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3401f182-6425-4c7b-9e7b-a7c99c39ee89 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
34772025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
34782025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
34792025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
34802025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34812025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34822025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34832025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
34842025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
34852025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
34862025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34872025-03-20T22:03:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
34882025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34892025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
34902025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
34912025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
34922025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
34932025-03-20T22:03:05.238ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
34942025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
34952025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34962025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34972025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
34982025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
34992025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
35002025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
35012025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
35022025-03-20T22:03:05.238ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-20 21:53:05.238061797 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
35032025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
35042025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
35052025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35062025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
35072025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35082025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
35092025-03-20T22:03:05.238ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f
35102025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
35112025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.23\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f"}
35122025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
35132025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
35142025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35152025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35162025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
35172025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
35182025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
35192025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35202025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
35212025-03-20T22:03:05.238ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
35222025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
35232025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
35242025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
35252025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
35262025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
35272025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35282025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
35292025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
35302025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
35312025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
35322025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
35332025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
35342025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
35352025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
35362025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35372025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
35382025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
35392025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
35402025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35412025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
35422025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
35432025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
35442025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
35452025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35462025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35472025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35482025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
35492025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
35502025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
35512025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35522025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
35532025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
35542025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
35552025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35562025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35572025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35582025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
35592025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35602025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
35612025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35622025-03-20T22:03:05.238ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55400d62-c4af-483b-8223-1234d14cf0ad.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3729, tv_nsec: 45194245 } })
35632025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35642025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35652025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35662025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35672025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
35682025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
35692025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
35702025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35712025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35722025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
35732025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35742025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35752025-03-20T22:03:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
35762025-03-20T22:03:05.238ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58723511-3cb5-4c26-bebf-16620d9cd641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("59acd018-3b3d-4cb6-bd0f-2dfaf2b8d965.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 3729, tv_nsec: 45367160 } })
35772025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35782025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35792025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
35802025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.067729ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
35812025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35822025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
35832025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
35842025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
35852025-03-20T22:03:05.239ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.178111ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
35862025-03-20T22:03:05.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35872025-03-20T22:03:05.241ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.015427ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
35882025-03-20T22:03:05.242ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.300985ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
35892025-03-20T22:03:05.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35902025-03-20T22:03:05.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35912025-03-20T22:03:05.244ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35922025-03-20T22:03:05.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35932025-03-20T22:03:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35942025-03-20T22:03:05.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35952025-03-20T22:03:05.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35962025-03-20T22:03:05.247ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-20 21:53:05.238061797 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
35972025-03-20T22:03:05.247ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.062322ms iteration = 13 status = {"expiration":"2025-03-20T21:53:05.238061797Z","pruned":[]}
35982025-03-20T22:03:05.247ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3729, tv_nsec: 54055692 } })
35992025-03-20T22:03:05.247ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
36002025-03-20T22:03:05.249ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36012025-03-20T22:03:05.250ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36022025-03-20T22:03:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
36032025-03-20T22:03:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
36042025-03-20T22:03:05.254ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
36052025-03-20T22:03:05.254ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
36062025-03-20T22:03:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
36072025-03-20T22:03:05.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36082025-03-20T22:03:05.261ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 23.222061ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
36092025-03-20T22:03:05.266ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36102025-03-20T22:03:05.267ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36112025-03-20T22:03:05.268ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36122025-03-20T22:03:05.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36132025-03-20T22:03:05.268ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36142025-03-20T22:03:05.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36152025-03-20T22:03:05.270ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36162025-03-20T22:03:05.270ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36172025-03-20T22:03:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
36182025-03-20T22:03:05.270ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
36192025-03-20T22:03:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36202025-03-20T22:03:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
36212025-03-20T22:03:05.278ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3729, tv_nsec: 84619630 } })
36222025-03-20T22:03:05.278ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
36232025-03-20T22:03:05.285ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36242025-03-20T22:03:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
36252025-03-20T22:03:05.289ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36262025-03-20T22:03:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36272025-03-20T22:03:05.290ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36282025-03-20T22:03:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
36292025-03-20T22:03:05.293ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36302025-03-20T22:03:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36312025-03-20T22:03:05.294ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36322025-03-20T22:03:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
36332025-03-20T22:03:05.297ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36342025-03-20T22:03:05.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36352025-03-20T22:03:05.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36362025-03-20T22:03:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36372025-03-20T22:03:05.302ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36382025-03-20T22:03:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36392025-03-20T22:03:05.306ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
36402025-03-20T22:03:05.306ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
36412025-03-20T22:03:05.306ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 68.81301ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
36422025-03-20T22:03:05.308ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36432025-03-20T22:03:05.309ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 71.470002ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
36442025-03-20T22:03:05.309ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36452025-03-20T22:03:05.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36462025-03-20T22:03:05.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36472025-03-20T22:03:05.312ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36482025-03-20T22:03:05.312ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36492025-03-20T22:03:05.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36502025-03-20T22:03:05.313ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36512025-03-20T22:03:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36522025-03-20T22:03:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
36532025-03-20T22:03:05.314ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36542025-03-20T22:03:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
36552025-03-20T22:03:05.316ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
36562025-03-20T22:03:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
36572025-03-20T22:03:05.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36582025-03-20T22:03:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36592025-03-20T22:03:05.317ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
36602025-03-20T22:03:05.317ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36612025-03-20T22:03:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
36622025-03-20T22:03:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
36632025-03-20T22:03:05.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36642025-03-20T22:03:05.319ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
36652025-03-20T22:03:05.319ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
36662025-03-20T22:03:05.319ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
36672025-03-20T22:03:05.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36682025-03-20T22:03:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36692025-03-20T22:03:05.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
36702025-03-20T22:03:05.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36712025-03-20T22:03:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36722025-03-20T22:03:05.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36732025-03-20T22:03:05.324ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36742025-03-20T22:03:05.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
36752025-03-20T22:03:05.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36762025-03-20T22:03:05.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36772025-03-20T22:03:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
36782025-03-20T22:03:05.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36792025-03-20T22:03:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36802025-03-20T22:03:05.333ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36812025-03-20T22:03:05.334ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36822025-03-20T22:03:05.335ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36832025-03-20T22:03:05.335ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
36842025-03-20T22:03:05.335ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
36852025-03-20T22:03:05.335ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36862025-03-20T22:03:05.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
36872025-03-20T22:03:05.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36882025-03-20T22:03:05.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
36892025-03-20T22:03:05.338ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36902025-03-20T22:03:05.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36912025-03-20T22:03:05.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
36922025-03-20T22:03:05.339ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36932025-03-20T22:03:05.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36942025-03-20T22:03:05.341ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
36952025-03-20T22:03:05.347ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36962025-03-20T22:03:05.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36972025-03-20T22:03:05.350ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36982025-03-20T22:03:05.353ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36992025-03-20T22:03:05.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
37002025-03-20T22:03:05.355ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37012025-03-20T22:03:05.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37022025-03-20T22:03:05.357ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37032025-03-20T22:03:05.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 121.008317ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
37042025-03-20T22:03:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
37052025-03-20T22:03:05.359ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
37062025-03-20T22:03:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
37072025-03-20T22:03:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
37082025-03-20T22:03:05.361ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37092025-03-20T22:03:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37102025-03-20T22:03:05.368ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3729, tv_nsec: 175216373 } })
37112025-03-20T22:03:05.369ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
37122025-03-20T22:03:05.369ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
37132025-03-20T22:03:05.369ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
37142025-03-20T22:03:05.369ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 131.649921ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
37152025-03-20T22:03:05.370ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
37162025-03-20T22:03:05.370ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 132.35654ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
37172025-03-20T22:03:05.370ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37182025-03-20T22:03:05.370ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
37192025-03-20T22:03:05.370ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37202025-03-20T22:03:05.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37212025-03-20T22:03:05.371ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37222025-03-20T22:03:05.371ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 133.322166ms iteration = 6 status = {"nsuccess":5}
37232025-03-20T22:03:05.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37242025-03-20T22:03:05.372ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
37252025-03-20T22:03:05.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
37262025-03-20T22:03:05.374ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
37272025-03-20T22:03:05.374ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 136.46409ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
37282025-03-20T22:03:05.375ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
37292025-03-20T22:03:05.375ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
37302025-03-20T22:03:05.375ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
37312025-03-20T22:03:05.375ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37322025-03-20T22:03:05.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37332025-03-20T22:03:05.377ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37342025-03-20T22:03:05.382ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37352025-03-20T22:03:05.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37362025-03-20T22:03:05.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 146.723815ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
37372025-03-20T22:03:05.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 146.893829ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
37382025-03-20T22:03:05.384ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37392025-03-20T22:03:05.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
37402025-03-20T22:03:05.386ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37412025-03-20T22:03:05.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37422025-03-20T22:03:05.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
37432025-03-20T22:03:05.387ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37442025-03-20T22:03:05.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37452025-03-20T22:03:05.388ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37462025-03-20T22:03:05.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37472025-03-20T22:03:05.390ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
37482025-03-20T22:03:05.390ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 152.114048ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
37492025-03-20T22:03:05.393ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
37502025-03-20T22:03:05.399ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
37512025-03-20T22:03:05.399ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37522025-03-20T22:03:05.399ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a855c668-04e1-4847-826f-00adbf37bbb9", "content-length": "26", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
37532025-03-20T22:03:05.399ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37542025-03-20T22:03:05.402ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37552025-03-20T22:03:05.403ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37562025-03-20T22:03:05.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37572025-03-20T22:03:05.415ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37582025-03-20T22:03:05.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37592025-03-20T22:03:05.420ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 182.391349ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37602025-03-20T22:03:05.423ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
37612025-03-20T22:03:05.423ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37622025-03-20T22:03:05.423ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9ef4de6-5028-4692-b748-7276bd58188c", "content-length": "26", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
37632025-03-20T22:03:05.423ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37642025-03-20T22:03:05.423ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37652025-03-20T22:03:05.423ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
37662025-03-20T22:03:05.423ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 185.546383ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
37672025-03-20T22:03:05.426ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37682025-03-20T22:03:05.429ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
37692025-03-20T22:03:05.429ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37702025-03-20T22:03:05.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37712025-03-20T22:03:05.430ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37722025-03-20T22:03:05.432ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 194.376649ms iteration = 25 status = 0
37732025-03-20T22:03:05.436ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
37742025-03-20T22:03:05.436ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
37752025-03-20T22:03:05.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37762025-03-20T22:03:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37772025-03-20T22:03:05.438ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
37782025-03-20T22:03:05.438ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 0 version = 2
37792025-03-20T22:03:05.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37802025-03-20T22:03:05.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37812025-03-20T22:03:05.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37822025-03-20T22:03:05.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37832025-03-20T22:03:05.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
37842025-03-20T22:03:05.442ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37852025-03-20T22:03:05.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37862025-03-20T22:03:05.443ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37872025-03-20T22:03:05.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37882025-03-20T22:03:05.444ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37892025-03-20T22:03:05.445ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
37902025-03-20T22:03:05.445ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
37912025-03-20T22:03:05.446ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-20 21:51:06.492803 UTC dns_group = internal generation = 1
37922025-03-20T22:03:05.446ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 207.792378ms iteration = 13 status = {"generation":1}
37932025-03-20T22:03:05.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
37942025-03-20T22:03:05.446ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37952025-03-20T22:03:05.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37962025-03-20T22:03:05.447ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
37972025-03-20T22:03:05.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37982025-03-20T22:03:05.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37992025-03-20T22:03:05.448ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 1 version = 2
38002025-03-20T22:03:05.448ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
38012025-03-20T22:03:05.448ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-20 21:51:07.691085 UTC dns_group = external generation = 2
38022025-03-20T22:03:05.448ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 210.12448ms iteration = 13 status = {"generation":2}
38032025-03-20T22:03:05.449ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 211.364083ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
38042025-03-20T22:03:05.472ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38052025-03-20T22:03:05.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38062025-03-20T22:03:05.475ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
38072025-03-20T22:03:05.475ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 236.913916ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
38082025-03-20T22:03:05.491ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-20T21:51:01.993052Z, time_modified: 2025-03-20T21:51:01.993052Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }) }
38092025-03-20T22:03:05.491ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 253.365576ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]}},"default_endpoint":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
38102025-03-20T22:03:05.491ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 253.866469ms iteration = 28 status = {"physical_disks_added":0}
38112025-03-20T22:03:05.500ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
38122025-03-20T22:03:05.510ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38132025-03-20T22:03:05.514ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38142025-03-20T22:03:05.514ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38152025-03-20T22:03:05.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38162025-03-20T22:03:05.561ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
38172025-03-20T22:03:05.562ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "1f005dd0-e9c2-427f-a873-83ccd0895e1d", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38182025-03-20T22:03:05.562ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38192025-03-20T22:03:05.564ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "2ddcf31e-fccc-457c-952b-5ef676254971", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38202025-03-20T22:03:05.564ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38212025-03-20T22:03:05.567ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38222025-03-20T22:03:05.567ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 328.909746ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
38232025-03-20T22:03:05.583ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
38242025-03-20T22:03:05.583ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38252025-03-20T22:03:05.583ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e541d91b-2990-47e8-ae22-b68097be8931", "content-length": "26", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38262025-03-20T22:03:05.583ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38272025-03-20T22:03:05.586ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
38282025-03-20T22:03:05.598ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 360.642945ms iteration = 25 status = {}
38292025-03-20T22:03:05.604ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
38302025-03-20T22:03:05.605ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b21e194d-f88a-415d-a464-9b7d4fd7153d", "content-length": "2", "date": "Thu, 20 Mar 2025 22:03:04 GMT"} })
38312025-03-20T22:03:05.606ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
38322025-03-20T22:03:05.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
38332025-03-20T22:03:05.628ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
38342025-03-20T22:03:05.628ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38352025-03-20T22:03:05.628ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d4da7c7-acd8-4434-88b1-4dca936cb651", "content-length": "372", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38362025-03-20T22:03:05.628ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38372025-03-20T22:03:05.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38382025-03-20T22:03:05.630ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38392025-03-20T22:03:05.630ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
38402025-03-20T22:03:05.630ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 392.872968ms iteration = 25 status = null
38412025-03-20T22:03:05.631ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "a2b86221-e814-4620-a6f2-c3a7aa09a03e", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
38422025-03-20T22:03:05.631ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
38432025-03-20T22:03:05.632ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "bf4e34ef-1282-44db-8fa3-77257bac1232", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
38442025-03-20T22:03:05.632ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
38452025-03-20T22:03:05.633ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38462025-03-20T22:03:05.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38472025-03-20T22:03:05.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38482025-03-20T22:03:05.634ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "792aa6b6-479b-48b6-8534-dec64fc2875b", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
38492025-03-20T22:03:05.634ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
38502025-03-20T22:03:05.636ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
38512025-03-20T22:03:05.636ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 398.905949ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
38522025-03-20T22:03:05.687ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
38532025-03-20T22:03:05.687ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
38542025-03-20T22:03:05.688ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c44084f1-6586-4acb-b6e6-d8a04cdb7ab3", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38552025-03-20T22:03:05.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38562025-03-20T22:03:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38572025-03-20T22:03:05.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38582025-03-20T22:03:05.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38592025-03-20T22:03:05.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38602025-03-20T22:03:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38612025-03-20T22:03:05.697ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
38622025-03-20T22:03:05.697ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38632025-03-20T22:03:05.697ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "74f36603-7e46-433b-a23c-61c5a9b54287", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38642025-03-20T22:03:05.699ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 461.315288ms iteration = 26 status = {}
38652025-03-20T22:03:05.723ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38662025-03-20T22:03:05.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38672025-03-20T22:03:05.725ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
38682025-03-20T22:03:05.725ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38692025-03-20T22:03:05.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38702025-03-20T22:03:05.735ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
38712025-03-20T22:03:05.735ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bd973e3-2406-474a-b22a-9681cbc18cf1", "content-length": "134", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38722025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
38732025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
38742025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
38752025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
38762025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38772025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38782025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38792025-03-20T22:03:05.735ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
38802025-03-20T22:03:05.736ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
38812025-03-20T22:03:05.736ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
38822025-03-20T22:03:05.736ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
38832025-03-20T22:03:05.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38842025-03-20T22:03:05.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38852025-03-20T22:03:05.742ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
38862025-03-20T22:03:05.742ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9444976-1967-41df-ae1c-a15957a190fd", "content-length": "691", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38872025-03-20T22:03:05.742ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
38882025-03-20T22:03:05.742ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
38892025-03-20T22:03:05.742ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
38902025-03-20T22:03:05.743ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 505.47599ms iteration = 26 status = {}
38912025-03-20T22:03:05.743ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
38922025-03-20T22:03:05.743ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fcb5bbf-540a-4f52-9be9-47a92fdef0e5", "content-length": "2", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38932025-03-20T22:03:05.743ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
38942025-03-20T22:03:05.743ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "af297a28-3564-4367-8d37-62063ab12e72", "content-length": "52", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38952025-03-20T22:03:05.743ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
38962025-03-20T22:03:05.744ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
38972025-03-20T22:03:05.832ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6440da5b-f8ce-4d6a-8ef8-9568cacaac26", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
38982025-03-20T22:03:05.832ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
38992025-03-20T22:03:05.832ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
39002025-03-20T22:03:05.836ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "23847c49-dd9d-43cd-8e11-89aaeb789197", "content-length": "682", "date": "Thu, 20 Mar 2025 22:03:05 GMT"} })
39012025-03-20T22:03:05.836ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39022025-03-20T22:03:05.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39032025-03-20T22:03:05.840ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39042025-03-20T22:03:05.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39052025-03-20T22:03:05.843ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39062025-03-20T22:03:05.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39072025-03-20T22:03:05.845ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39082025-03-20T22:03:05.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39092025-03-20T22:03:05.852ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 614.202538ms iteration = 25 status = {}
39102025-03-20T22:03:05.982ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 79267e92-5b84-419d-b0f8-a7f3178e5e6f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39112025-03-20T22:03:05.982ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 79267e92-5b84-419d-b0f8-a7f3178e5e6f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39122025-03-20T22:03:05.984ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2205 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 79267e92-5b84-419d-b0f8-a7f3178e5e6f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39132025-03-20T22:03:07.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5ebb8d16-14ad-408e-b298-4f429ba3185e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39142025-03-20T22:03:07.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5ebb8d16-14ad-408e-b298-4f429ba3185e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39152025-03-20T22:03:07.007ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4956 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5ebb8d16-14ad-408e-b298-4f429ba3185e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39162025-03-20T22:03:07.254ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 570 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 7c9c833b-051e-42c3-bf52-4478d41b73f7 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
39172025-03-20T22:03:08.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0f8b87f2-407a-4ae7-8df2-fce145bc7ec8 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39182025-03-20T22:03:08.027ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0f8b87f2-407a-4ae7-8df2-fce145bc7ec8 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39192025-03-20T22:03:08.029ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2408 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0f8b87f2-407a-4ae7-8df2-fce145bc7ec8 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39202025-03-20T22:03:09.048ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4cb9597c-d2d9-466a-988e-9ecfefecd24a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39212025-03-20T22:03:09.048ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4cb9597c-d2d9-466a-988e-9ecfefecd24a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39222025-03-20T22:03:09.051ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2627 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4cb9597c-d2d9-466a-988e-9ecfefecd24a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39232025-03-20T22:03:10.071ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 02793263-0382-4505-b0e9-daf15ccdc82f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39242025-03-20T22:03:10.072ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 02793263-0382-4505-b0e9-daf15ccdc82f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39252025-03-20T22:03:10.074ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2603 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 02793263-0382-4505-b0e9-daf15ccdc82f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39262025-03-20T22:03:11.093ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fbd27de4-433e-4386-96c7-940c50c2bb8d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39272025-03-20T22:03:11.093ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fbd27de4-433e-4386-96c7-940c50c2bb8d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39282025-03-20T22:03:11.095ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1932 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fbd27de4-433e-4386-96c7-940c50c2bb8d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39292025-03-20T22:03:12.113ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f8cde161-a090-458a-9316-5f18a435fa5c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39302025-03-20T22:03:12.113ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f8cde161-a090-458a-9316-5f18a435fa5c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39312025-03-20T22:03:12.115ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2408 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f8cde161-a090-458a-9316-5f18a435fa5c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39322025-03-20T22:03:13.134ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9d3974b6-a780-4b84-b3b5-d865423d357f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39332025-03-20T22:03:13.134ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9d3974b6-a780-4b84-b3b5-d865423d357f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39342025-03-20T22:03:13.136ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2464 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9d3974b6-a780-4b84-b3b5-d865423d357f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39352025-03-20T22:03:14.155ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3b172332-f626-41d4-a047-d2da8da95175 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39362025-03-20T22:03:14.156ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3b172332-f626-41d4-a047-d2da8da95175 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39372025-03-20T22:03:14.158ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2330 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3b172332-f626-41d4-a047-d2da8da95175 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39382025-03-20T22:03:15.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e43005c5-b041-4e08-93d1-816eaa6dd5cb roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39392025-03-20T22:03:15.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e43005c5-b041-4e08-93d1-816eaa6dd5cb resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39402025-03-20T22:03:15.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2272 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e43005c5-b041-4e08-93d1-816eaa6dd5cb response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39412025-03-20T22:03:15.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
39422025-03-20T22:03:15.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
39432025-03-20T22:03:15.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39442025-03-20T22:03:15.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39452025-03-20T22:03:15.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39462025-03-20T22:03:15.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39472025-03-20T22:03:15.270ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39482025-03-20T22:03:15.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39492025-03-20T22:03:15.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39502025-03-20T22:03:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39512025-03-20T22:03:15.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39522025-03-20T22:03:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39532025-03-20T22:03:15.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39542025-03-20T22:03:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39552025-03-20T22:03:15.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39562025-03-20T22:03:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
39572025-03-20T22:03:15.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39582025-03-20T22:03:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39592025-03-20T22:03:15.293ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
39602025-03-20T22:03:15.293ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 55.389812ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
39612025-03-20T22:03:15.301ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.015962ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39622025-03-20T22:03:16.198ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f4dfe183-3211-466e-85b3-16571b2c68cd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39632025-03-20T22:03:16.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f4dfe183-3211-466e-85b3-16571b2c68cd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39642025-03-20T22:03:16.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1899 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f4dfe183-3211-466e-85b3-16571b2c68cd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39652025-03-20T22:03:17.219ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2302a5d9-c417-4248-8f93-65991dd2ce09 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39662025-03-20T22:03:17.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2302a5d9-c417-4248-8f93-65991dd2ce09 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39672025-03-20T22:03:17.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2281 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2302a5d9-c417-4248-8f93-65991dd2ce09 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39682025-03-20T22:03:17.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = ec2e125c-456a-49bc-a167-da8a92827e1f roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
39692025-03-20T22:03:17.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = ec2e125c-456a-49bc-a167-da8a92827e1f resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
39702025-03-20T22:03:17.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3085 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = ec2e125c-456a-49bc-a167-da8a92827e1f response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
39712025-03-20T22:03:17.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b5445b64-7152-432c-89b3-e1af1cedad68 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
39722025-03-20T22:03:17.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b5445b64-7152-432c-89b3-e1af1cedad68 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
39732025-03-20T22:03:17.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2021 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b5445b64-7152-432c-89b3-e1af1cedad68 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
39742025-03-20T22:03:17.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 618 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 7fbd55ea-2334-43b7-8ebf-40c2c130a7e6 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
39752025-03-20T22:03:18.242ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1a332349-a630-4bd3-9d32-c3aea643afc7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39762025-03-20T22:03:18.242ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1a332349-a630-4bd3-9d32-c3aea643afc7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39772025-03-20T22:03:18.243ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1983 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1a332349-a630-4bd3-9d32-c3aea643afc7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39782025-03-20T22:03:19.263ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0f96b4b-191b-44a6-9316-bed7a5fe680f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39792025-03-20T22:03:19.264ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0f96b4b-191b-44a6-9316-bed7a5fe680f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39802025-03-20T22:03:19.266ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2649 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e0f96b4b-191b-44a6-9316-bed7a5fe680f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39812025-03-20T22:03:20.285ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c2d6adf-88d0-4499-aa42-6cde07703b29 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39822025-03-20T22:03:20.286ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c2d6adf-88d0-4499-aa42-6cde07703b29 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39832025-03-20T22:03:20.288ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2797 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5c2d6adf-88d0-4499-aa42-6cde07703b29 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39842025-03-20T22:03:21.308ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8744bb22-0e55-4a6d-8ab9-a3f093c49726 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39852025-03-20T22:03:21.309ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8744bb22-0e55-4a6d-8ab9-a3f093c49726 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39862025-03-20T22:03:21.311ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2493 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8744bb22-0e55-4a6d-8ab9-a3f093c49726 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39872025-03-20T22:03:22.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8bc3b9ad-0b98-4a2d-9f65-c81a8cf689c7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39882025-03-20T22:03:22.332ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8bc3b9ad-0b98-4a2d-9f65-c81a8cf689c7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39892025-03-20T22:03:22.334ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2495 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8bc3b9ad-0b98-4a2d-9f65-c81a8cf689c7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39902025-03-20T22:03:23.353ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c90f5145-5f31-4179-b8f2-b5a98feea185 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39912025-03-20T22:03:23.353ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c90f5145-5f31-4179-b8f2-b5a98feea185 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39922025-03-20T22:03:23.355ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2159 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c90f5145-5f31-4179-b8f2-b5a98feea185 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39932025-03-20T22:03:24.375ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f01b6139-0a25-46c3-8c08-42e28aed804c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39942025-03-20T22:03:24.375ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f01b6139-0a25-46c3-8c08-42e28aed804c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39952025-03-20T22:03:24.377ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2355 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f01b6139-0a25-46c3-8c08-42e28aed804c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
39962025-03-20T22:03:25.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
39972025-03-20T22:03:25.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
39982025-03-20T22:03:25.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39992025-03-20T22:03:25.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40002025-03-20T22:03:25.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40012025-03-20T22:03:25.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40022025-03-20T22:03:25.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40032025-03-20T22:03:25.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40042025-03-20T22:03:25.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40052025-03-20T22:03:25.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40062025-03-20T22:03:25.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40072025-03-20T22:03:25.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40082025-03-20T22:03:25.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40092025-03-20T22:03:25.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40102025-03-20T22:03:25.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40112025-03-20T22:03:25.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
40122025-03-20T22:03:25.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40132025-03-20T22:03:25.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40142025-03-20T22:03:25.256ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
40152025-03-20T22:03:25.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.287095ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
40162025-03-20T22:03:25.268ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.573445ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40172025-03-20T22:03:25.395ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 919c0669-d245-465e-98fc-7ec17b8ea82a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40182025-03-20T22:03:25.395ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 919c0669-d245-465e-98fc-7ec17b8ea82a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40192025-03-20T22:03:25.397ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2258 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 919c0669-d245-465e-98fc-7ec17b8ea82a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40202025-03-20T22:03:26.418ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c6772af2-47ea-4a8e-b8e3-8c92377c398a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40212025-03-20T22:03:26.418ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c6772af2-47ea-4a8e-b8e3-8c92377c398a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40222025-03-20T22:03:26.420ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c6772af2-47ea-4a8e-b8e3-8c92377c398a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40232025-03-20T22:03:27.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 530 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 6b65570d-9e33-43fc-9632-92242d947498 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
40242025-03-20T22:03:27.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5ef0cf6b-acbb-494e-a96a-8c4d9fe0545d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40252025-03-20T22:03:27.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5ef0cf6b-acbb-494e-a96a-8c4d9fe0545d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40262025-03-20T22:03:27.441ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1983 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5ef0cf6b-acbb-494e-a96a-8c4d9fe0545d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40272025-03-20T22:03:28.461ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = de3d23f1-ad32-4169-9683-e7b7eac29ebd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40282025-03-20T22:03:28.462ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = de3d23f1-ad32-4169-9683-e7b7eac29ebd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40292025-03-20T22:03:28.464ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2492 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = de3d23f1-ad32-4169-9683-e7b7eac29ebd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40302025-03-20T22:03:29.483ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 694672e6-cd9b-4983-a1e5-44a85ce6c27f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40312025-03-20T22:03:29.483ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 694672e6-cd9b-4983-a1e5-44a85ce6c27f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40322025-03-20T22:03:29.486ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2865 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 694672e6-cd9b-4983-a1e5-44a85ce6c27f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40332025-03-20T22:03:30.506ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6fdbf082-3965-4920-ac7f-fd2ccc3d34e6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40342025-03-20T22:03:30.507ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6fdbf082-3965-4920-ac7f-fd2ccc3d34e6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40352025-03-20T22:03:30.509ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2294 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 6fdbf082-3965-4920-ac7f-fd2ccc3d34e6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40362025-03-20T22:03:31.528ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5a66d4db-91d5-418f-bc3e-7b5e18fe4675 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40372025-03-20T22:03:31.529ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5a66d4db-91d5-418f-bc3e-7b5e18fe4675 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40382025-03-20T22:03:31.531ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2679 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5a66d4db-91d5-418f-bc3e-7b5e18fe4675 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40392025-03-20T22:03:32.247ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = dfc773b8-0762-4380-8bfe-2f6ae9266666 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
40402025-03-20T22:03:32.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = dfc773b8-0762-4380-8bfe-2f6ae9266666 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
40412025-03-20T22:03:32.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3253 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = dfc773b8-0762-4380-8bfe-2f6ae9266666 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
40422025-03-20T22:03:32.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b63b3d9b-883d-4136-aa4f-dd2e667dee5b roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
40432025-03-20T22:03:32.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b63b3d9b-883d-4136-aa4f-dd2e667dee5b resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
40442025-03-20T22:03:32.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2618 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = b63b3d9b-883d-4136-aa4f-dd2e667dee5b response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
40452025-03-20T22:03:32.552ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a781ffcb-5746-4bf0-a057-d31f5956d751 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40462025-03-20T22:03:32.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a781ffcb-5746-4bf0-a057-d31f5956d751 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40472025-03-20T22:03:32.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a781ffcb-5746-4bf0-a057-d31f5956d751 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40482025-03-20T22:03:33.575ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 586bc9bf-ed11-478e-99c8-b4c30c909755 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40492025-03-20T22:03:33.575ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 586bc9bf-ed11-478e-99c8-b4c30c909755 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40502025-03-20T22:03:33.577ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2192 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 586bc9bf-ed11-478e-99c8-b4c30c909755 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40512025-03-20T22:03:34.597ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 499d22d5-8c50-445f-9887-96a5971eb1a8 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40522025-03-20T22:03:34.598ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 499d22d5-8c50-445f-9887-96a5971eb1a8 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40532025-03-20T22:03:34.600ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2343 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 499d22d5-8c50-445f-9887-96a5971eb1a8 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
40542025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
40552025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
40562025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40572025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40582025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
40592025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
40602025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
40612025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40622025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
40632025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
40642025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40652025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40662025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
40672025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
40682025-03-20T22:03:35.238ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
40692025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
40702025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40712025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
40722025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
40732025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
40742025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
40752025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
40762025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40772025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
40782025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
40792025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
40802025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
40812025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
40822025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40832025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
40842025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
40852025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40862025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40872025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
40882025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40892025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
40902025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
40912025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
40922025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40932025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40942025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40952025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40962025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40972025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40982025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
40992025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41002025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41012025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
41022025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
41032025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41042025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41052025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
41062025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
41072025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41082025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41092025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41102025-03-20T22:03:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41112025-03-20T22:03:35.242ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41122025-03-20T22:03:35.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41132025-03-20T22:03:35.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
41142025-03-20T22:03:35.245ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41152025-03-20T22:03:35.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41162025-03-20T22:03:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41172025-03-20T22:03:35.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41182025-03-20T22:03:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41192025-03-20T22:03:35.248ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3759, tv_nsec: 55091600 } })
41202025-03-20T22:03:35.248ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.066859ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
41212025-03-20T22:03:35.248ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41222025-03-20T22:03:35.248ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41232025-03-20T22:03:35.248ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
41242025-03-20T22:03:35.249ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41252025-03-20T22:03:35.250ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.745474ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
41262025-03-20T22:03:35.251ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
41272025-03-20T22:03:35.251ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41282025-03-20T22:03:35.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41292025-03-20T22:03:35.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
41302025-03-20T22:03:35.252ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41312025-03-20T22:03:35.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41322025-03-20T22:03:35.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
41332025-03-20T22:03:35.252ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41342025-03-20T22:03:35.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
41352025-03-20T22:03:35.256ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3759, tv_nsec: 63579117 } })
41362025-03-20T22:03:35.257ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
41372025-03-20T22:03:35.259ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41382025-03-20T22:03:35.260ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41392025-03-20T22:03:35.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
41402025-03-20T22:03:35.263ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41412025-03-20T22:03:35.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41422025-03-20T22:03:35.264ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
41432025-03-20T22:03:35.264ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41442025-03-20T22:03:35.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41452025-03-20T22:03:35.267ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41462025-03-20T22:03:35.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41472025-03-20T22:03:35.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41482025-03-20T22:03:35.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41492025-03-20T22:03:35.270ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3759, tv_nsec: 76726369 } })
41502025-03-20T22:03:35.270ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41512025-03-20T22:03:35.270ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41522025-03-20T22:03:35.270ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
41532025-03-20T22:03:35.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41542025-03-20T22:03:35.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
41552025-03-20T22:03:35.273ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41562025-03-20T22:03:35.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41572025-03-20T22:03:35.275ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
41582025-03-20T22:03:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 37.193385ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
41592025-03-20T22:03:35.302ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41602025-03-20T22:03:35.302ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41612025-03-20T22:03:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41622025-03-20T22:03:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41632025-03-20T22:03:35.304ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
41642025-03-20T22:03:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41652025-03-20T22:03:35.304ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.757106ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
41662025-03-20T22:03:35.305ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 67.29762ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
41672025-03-20T22:03:35.306ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41682025-03-20T22:03:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41692025-03-20T22:03:35.309ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 71.676707ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41702025-03-20T22:03:35.315ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 77.082522ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
41712025-03-20T22:03:35.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
41722025-03-20T22:03:35.320ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41732025-03-20T22:03:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41742025-03-20T22:03:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
41752025-03-20T22:03:35.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41762025-03-20T22:03:35.322ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41772025-03-20T22:03:35.322ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
41782025-03-20T22:03:35.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41792025-03-20T22:03:35.322ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41802025-03-20T22:03:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
41812025-03-20T22:03:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41822025-03-20T22:03:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
41832025-03-20T22:03:35.325ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41842025-03-20T22:03:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41852025-03-20T22:03:35.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
41862025-03-20T22:03:35.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41872025-03-20T22:03:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41882025-03-20T22:03:35.326ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41892025-03-20T22:03:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41902025-03-20T22:03:35.327ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
41912025-03-20T22:03:35.327ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 89.328099ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
41922025-03-20T22:03:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
41932025-03-20T22:03:35.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41942025-03-20T22:03:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41952025-03-20T22:03:35.331ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
41962025-03-20T22:03:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
41972025-03-20T22:03:35.333ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41982025-03-20T22:03:35.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
41992025-03-20T22:03:35.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42002025-03-20T22:03:35.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42012025-03-20T22:03:35.339ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 100.801727ms iteration = 26 status = 0
42022025-03-20T22:03:35.339ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
42032025-03-20T22:03:35.339ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 101.450594ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
42042025-03-20T22:03:35.341ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 102.937973ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
42052025-03-20T22:03:35.341ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
42062025-03-20T22:03:35.341ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42072025-03-20T22:03:35.341ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3531ca51-6d7a-4c42-8f60-e1ad68367153", "content-length": "26", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42082025-03-20T22:03:35.341ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42092025-03-20T22:03:35.344ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42102025-03-20T22:03:35.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 120.499893ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
42112025-03-20T22:03:35.375ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42122025-03-20T22:03:35.375ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
42132025-03-20T22:03:35.375ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42142025-03-20T22:03:35.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
42152025-03-20T22:03:35.379ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
42162025-03-20T22:03:35.379ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42172025-03-20T22:03:35.379ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "96c60b25-a4c3-49b8-be08-a0603d786636", "content-length": "26", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42182025-03-20T22:03:35.379ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42192025-03-20T22:03:35.381ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
42202025-03-20T22:03:35.385ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
42212025-03-20T22:03:35.385ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
42222025-03-20T22:03:35.385ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cc47dfe-0830-4ab9-9be2-5e5d48fff05d", "content-length": "26", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42232025-03-20T22:03:35.385ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
42242025-03-20T22:03:35.387ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
42252025-03-20T22:03:35.392ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42262025-03-20T22:03:35.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42272025-03-20T22:03:35.405ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 167.49423ms iteration = 29 status = {"physical_disks_added":0}
42282025-03-20T22:03:35.416ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42292025-03-20T22:03:35.417ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
42302025-03-20T22:03:35.417ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 178.572257ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
42312025-03-20T22:03:35.428ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
42322025-03-20T22:03:35.428ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42332025-03-20T22:03:35.429ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "183a69b8-65cc-4087-b14d-eb3c90bb45f6", "content-length": "372", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42342025-03-20T22:03:35.429ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42352025-03-20T22:03:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42362025-03-20T22:03:35.448ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 209.673778ms iteration = 26 status = {}
42372025-03-20T22:03:35.463ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
42382025-03-20T22:03:35.464ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d931670a-cbb5-46ab-894c-63321ca31f2d", "content-length": "2", "date": "Thu, 20 Mar 2025 22:03:34 GMT"} })
42392025-03-20T22:03:35.466ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
42402025-03-20T22:03:35.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
42412025-03-20T22:03:35.468ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 230.447074ms iteration = 26 status = null
42422025-03-20T22:03:35.473ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
42432025-03-20T22:03:35.473ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
42442025-03-20T22:03:35.473ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d906ec88-010a-4281-88e9-67db883220e0", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42452025-03-20T22:03:35.473ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42462025-03-20T22:03:35.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42472025-03-20T22:03:35.475ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42482025-03-20T22:03:35.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42492025-03-20T22:03:35.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42502025-03-20T22:03:35.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42512025-03-20T22:03:35.481ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
42522025-03-20T22:03:35.481ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42532025-03-20T22:03:35.482ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b247ba73-bf18-4651-b9f8-eed75ae42836", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42542025-03-20T22:03:35.484ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 245.906917ms iteration = 27 status = {}
42552025-03-20T22:03:35.485ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42562025-03-20T22:03:35.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42572025-03-20T22:03:35.487ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42582025-03-20T22:03:35.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42592025-03-20T22:03:35.489ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42602025-03-20T22:03:35.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42612025-03-20T22:03:35.491ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42622025-03-20T22:03:35.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42632025-03-20T22:03:35.493ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42642025-03-20T22:03:35.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42652025-03-20T22:03:35.548ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42662025-03-20T22:03:35.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42672025-03-20T22:03:35.558ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
42682025-03-20T22:03:35.558ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42692025-03-20T22:03:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42702025-03-20T22:03:35.571ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
42712025-03-20T22:03:35.571ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "987341ba-ab85-4cee-9c8a-df675866f7a8", "content-length": "134", "date": "Thu, 20 Mar 2025 22:03:34 GMT"} })
42722025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
42732025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
42742025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
42752025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
42762025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42772025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42782025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42792025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42802025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
42812025-03-20T22:03:35.571ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
42822025-03-20T22:03:35.572ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
42832025-03-20T22:03:35.572ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42842025-03-20T22:03:35.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42852025-03-20T22:03:35.574ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
42862025-03-20T22:03:35.574ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "24c1cf54-a2b4-4694-b5b8-9c4f2934f40c", "content-length": "2", "date": "Thu, 20 Mar 2025 22:03:34 GMT"} })
42872025-03-20T22:03:35.574ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
42882025-03-20T22:03:35.574ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e556479b-1408-4bff-898c-a70a4db128a5", "content-length": "52", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42892025-03-20T22:03:35.574ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
42902025-03-20T22:03:35.574ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
42912025-03-20T22:03:35.617ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
42922025-03-20T22:03:35.676ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee421e11-a825-4a72-a94b-33b6befbfa27 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
42932025-03-20T22:03:35.677ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee421e11-a825-4a72-a94b-33b6befbfa27 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
42942025-03-20T22:03:35.679ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2556 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee421e11-a825-4a72-a94b-33b6befbfa27 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
42952025-03-20T22:03:35.686ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7032446d-08f6-4eeb-ae49-94f5d788104d", "content-length": "691", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
42962025-03-20T22:03:35.686ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
42972025-03-20T22:03:35.686ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
42982025-03-20T22:03:35.686ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
42992025-03-20T22:03:35.688ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 449.713918ms iteration = 27 status = {}
43002025-03-20T22:03:35.769ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8925c42b-c35e-4b6d-9cc5-be3fd5746410", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
43012025-03-20T22:03:35.769ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
43022025-03-20T22:03:35.769ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
43032025-03-20T22:03:35.769ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6428bf0a-882f-4b4c-904d-e9315a66daa9", "content-length": "682", "date": "Thu, 20 Mar 2025 22:03:35 GMT"} })
43042025-03-20T22:03:35.769ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43052025-03-20T22:03:35.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43062025-03-20T22:03:35.773ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43072025-03-20T22:03:35.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43082025-03-20T22:03:35.776ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43092025-03-20T22:03:35.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43102025-03-20T22:03:35.778ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43112025-03-20T22:03:35.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43122025-03-20T22:03:35.786ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 547.872114ms iteration = 26 status = {}
43132025-03-20T22:03:36.701ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bfd76c9a-8f9d-4682-ad84-fbe9fd213723 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43142025-03-20T22:03:36.702ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bfd76c9a-8f9d-4682-ad84-fbe9fd213723 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43152025-03-20T22:03:36.703ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1842 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bfd76c9a-8f9d-4682-ad84-fbe9fd213723 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43162025-03-20T22:03:37.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 547 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = d63bb098-4666-45e4-a815-3a1a3d89d66a response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
43172025-03-20T22:03:37.723ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2045aa9f-bb99-4edb-a138-475748512e25 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43182025-03-20T22:03:37.723ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2045aa9f-bb99-4edb-a138-475748512e25 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43192025-03-20T22:03:37.726ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2602 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 2045aa9f-bb99-4edb-a138-475748512e25 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43202025-03-20T22:03:38.745ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 36d01fd4-c7a6-4859-bba6-5cd970fdb002 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43212025-03-20T22:03:38.746ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 36d01fd4-c7a6-4859-bba6-5cd970fdb002 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43222025-03-20T22:03:38.748ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2458 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 36d01fd4-c7a6-4859-bba6-5cd970fdb002 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43232025-03-20T22:03:39.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 65509efd-4f2f-4e29-9531-e714343e78b1 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43242025-03-20T22:03:39.767ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 65509efd-4f2f-4e29-9531-e714343e78b1 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43252025-03-20T22:03:39.770ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2515 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 65509efd-4f2f-4e29-9531-e714343e78b1 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43262025-03-20T22:03:40.790ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4df22797-07ed-4ee8-b007-8edc071c93a5 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43272025-03-20T22:03:40.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4df22797-07ed-4ee8-b007-8edc071c93a5 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43282025-03-20T22:03:40.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2764 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4df22797-07ed-4ee8-b007-8edc071c93a5 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43292025-03-20T22:03:41.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 461ea767-cc82-4d71-9b71-2217ec33c1e0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43302025-03-20T22:03:41.812ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 461ea767-cc82-4d71-9b71-2217ec33c1e0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43312025-03-20T22:03:41.814ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2261 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 461ea767-cc82-4d71-9b71-2217ec33c1e0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43322025-03-20T22:03:42.833ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = aeaee389-00ec-42ca-aa3b-1971ae81982c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43332025-03-20T22:03:42.833ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = aeaee389-00ec-42ca-aa3b-1971ae81982c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43342025-03-20T22:03:42.835ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2318 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = aeaee389-00ec-42ca-aa3b-1971ae81982c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43352025-03-20T22:03:43.854ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 65db6cb1-70e1-4a75-ad23-c805aa1507c4 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43362025-03-20T22:03:43.854ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 65db6cb1-70e1-4a75-ad23-c805aa1507c4 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43372025-03-20T22:03:43.856ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2110 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 65db6cb1-70e1-4a75-ad23-c805aa1507c4 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43382025-03-20T22:03:44.875ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8483ba9b-574a-4a33-9fc7-7836033c5ed6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43392025-03-20T22:03:44.875ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8483ba9b-574a-4a33-9fc7-7836033c5ed6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43402025-03-20T22:03:44.877ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2460 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8483ba9b-574a-4a33-9fc7-7836033c5ed6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43412025-03-20T22:03:45.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
43422025-03-20T22:03:45.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
43432025-03-20T22:03:45.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43442025-03-20T22:03:45.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43452025-03-20T22:03:45.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43462025-03-20T22:03:45.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43472025-03-20T22:03:45.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43482025-03-20T22:03:45.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43492025-03-20T22:03:45.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43502025-03-20T22:03:45.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43512025-03-20T22:03:45.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43522025-03-20T22:03:45.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43532025-03-20T22:03:45.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43542025-03-20T22:03:45.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
43552025-03-20T22:03:45.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43562025-03-20T22:03:45.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43572025-03-20T22:03:45.259ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
43582025-03-20T22:03:45.259ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.503496ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
43592025-03-20T22:03:45.268ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43602025-03-20T22:03:45.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43612025-03-20T22:03:45.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.365726ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43622025-03-20T22:03:45.896ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 72ac0958-4283-489a-8d55-5e2233c5141c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43632025-03-20T22:03:45.897ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 72ac0958-4283-489a-8d55-5e2233c5141c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43642025-03-20T22:03:45.898ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2092 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 72ac0958-4283-489a-8d55-5e2233c5141c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43652025-03-20T22:03:46.918ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0e306bc0-30a7-4747-9b8c-d3628e912495 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43662025-03-20T22:03:46.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0e306bc0-30a7-4747-9b8c-d3628e912495 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43672025-03-20T22:03:46.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2336 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0e306bc0-30a7-4747-9b8c-d3628e912495 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43682025-03-20T22:03:47.252ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 537 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 570bf02f-28a0-4ed4-bda3-7ff5918c141f response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
43692025-03-20T22:03:47.939ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1b2d4356-d379-4a46-875d-b33dc14d639d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43702025-03-20T22:03:47.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1b2d4356-d379-4a46-875d-b33dc14d639d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43712025-03-20T22:03:47.941ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2332 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1b2d4356-d379-4a46-875d-b33dc14d639d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43722025-03-20T22:03:48.961ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7327c759-560d-420b-8a23-4fd441139d83 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43732025-03-20T22:03:48.962ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7327c759-560d-420b-8a23-4fd441139d83 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43742025-03-20T22:03:48.963ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2559 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7327c759-560d-420b-8a23-4fd441139d83 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43752025-03-20T22:03:49.984ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d740bdea-6a9a-4d73-9962-3bb52772838a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43762025-03-20T22:03:49.984ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d740bdea-6a9a-4d73-9962-3bb52772838a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43772025-03-20T22:03:49.986ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2273 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d740bdea-6a9a-4d73-9962-3bb52772838a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43782025-03-20T22:03:51.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 05a9a366-e236-4b46-bbcc-52f123aae26a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43792025-03-20T22:03:51.006ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 05a9a366-e236-4b46-bbcc-52f123aae26a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43802025-03-20T22:03:51.008ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2103 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 05a9a366-e236-4b46-bbcc-52f123aae26a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43812025-03-20T22:03:52.027ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 66438d69-afb4-430e-b0a0-72c0248b0d48 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43822025-03-20T22:03:52.028ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 66438d69-afb4-430e-b0a0-72c0248b0d48 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43832025-03-20T22:03:52.030ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2596 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 66438d69-afb4-430e-b0a0-72c0248b0d48 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43842025-03-20T22:03:53.049ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80c04c7a-c3dc-4176-a4bf-9e2534f29b8d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43852025-03-20T22:03:53.050ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80c04c7a-c3dc-4176-a4bf-9e2534f29b8d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43862025-03-20T22:03:53.052ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2644 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 80c04c7a-c3dc-4176-a4bf-9e2534f29b8d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43872025-03-20T22:03:54.071ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d308118f-6016-4d2b-820d-c50d2adba2c6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43882025-03-20T22:03:54.071ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d308118f-6016-4d2b-820d-c50d2adba2c6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43892025-03-20T22:03:54.073ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2088 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d308118f-6016-4d2b-820d-c50d2adba2c6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43902025-03-20T22:03:55.093ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62e314e5-e50e-411b-b7f3-51d11c0453dd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43912025-03-20T22:03:55.093ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62e314e5-e50e-411b-b7f3-51d11c0453dd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43922025-03-20T22:03:55.095ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2658 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 62e314e5-e50e-411b-b7f3-51d11c0453dd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
43932025-03-20T22:03:55.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
43942025-03-20T22:03:55.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
43952025-03-20T22:03:55.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43962025-03-20T22:03:55.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43972025-03-20T22:03:55.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43982025-03-20T22:03:55.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43992025-03-20T22:03:55.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44002025-03-20T22:03:55.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44012025-03-20T22:03:55.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44022025-03-20T22:03:55.241ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.938406ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44032025-03-20T22:03:55.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
44042025-03-20T22:03:55.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44052025-03-20T22:03:55.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44062025-03-20T22:03:55.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44072025-03-20T22:03:55.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44082025-03-20T22:03:55.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44092025-03-20T22:03:55.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
44102025-03-20T22:03:55.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44112025-03-20T22:03:55.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44122025-03-20T22:03:55.259ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
44132025-03-20T22:03:55.259ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.711461ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
44142025-03-20T22:03:56.115ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ac7a2920-c476-4e1c-8275-7a165a6a5462 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44152025-03-20T22:03:56.116ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ac7a2920-c476-4e1c-8275-7a165a6a5462 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44162025-03-20T22:03:56.117ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2451 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ac7a2920-c476-4e1c-8275-7a165a6a5462 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44172025-03-20T22:03:57.138ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a7907532-2728-4e77-a141-67737fa3906a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44182025-03-20T22:03:57.138ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a7907532-2728-4e77-a141-67737fa3906a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44192025-03-20T22:03:57.140ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2381 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a7907532-2728-4e77-a141-67737fa3906a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44202025-03-20T22:03:57.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 564 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 7b9b90ff-7245-49d9-b883-3bbc2e1f3f23 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
44212025-03-20T22:03:58.161ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f71c6960-4233-4730-b54d-fb52f177c7d3 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44222025-03-20T22:03:58.162ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f71c6960-4233-4730-b54d-fb52f177c7d3 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44232025-03-20T22:03:58.164ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2743 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f71c6960-4233-4730-b54d-fb52f177c7d3 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44242025-03-20T22:03:59.183ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a3ba9c7b-6924-4e44-9c1a-8a572d11aec8 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44252025-03-20T22:03:59.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a3ba9c7b-6924-4e44-9c1a-8a572d11aec8 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44262025-03-20T22:03:59.186ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2365 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a3ba9c7b-6924-4e44-9c1a-8a572d11aec8 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44272025-03-20T22:04:00.206ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1439790a-0b5a-4bc6-beaa-58230093f350 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44282025-03-20T22:04:00.207ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1439790a-0b5a-4bc6-beaa-58230093f350 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44292025-03-20T22:04:00.208ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2044 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1439790a-0b5a-4bc6-beaa-58230093f350 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44302025-03-20T22:04:01.229ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5cc31431-aad7-49b3-b6a5-63d0b2836e8d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44312025-03-20T22:04:01.229ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5cc31431-aad7-49b3-b6a5-63d0b2836e8d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44322025-03-20T22:04:01.231ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2248 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5cc31431-aad7-49b3-b6a5-63d0b2836e8d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44332025-03-20T22:04:02.246ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = d90af586-4957-454e-9932-449269f33caa roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
44342025-03-20T22:04:02.247ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = d90af586-4957-454e-9932-449269f33caa resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
44352025-03-20T22:04:02.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2953 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = d90af586-4957-454e-9932-449269f33caa response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
44362025-03-20T22:04:02.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 050dffa6-941d-40a6-9794-12ceba552607 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
44372025-03-20T22:04:02.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 050dffa6-941d-40a6-9794-12ceba552607 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
44382025-03-20T22:04:02.251ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22b8f84e-fce9-4f68-891d-5c6cf5c29e4c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44392025-03-20T22:04:02.251ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22b8f84e-fce9-4f68-891d-5c6cf5c29e4c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44402025-03-20T22:04:02.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2804 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 050dffa6-941d-40a6-9794-12ceba552607 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
44412025-03-20T22:04:02.253ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2004 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 22b8f84e-fce9-4f68-891d-5c6cf5c29e4c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44422025-03-20T22:04:03.273ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = afc016e8-505a-4b56-bb44-ed7150616716 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44432025-03-20T22:04:03.273ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = afc016e8-505a-4b56-bb44-ed7150616716 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44442025-03-20T22:04:03.274ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1897 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = afc016e8-505a-4b56-bb44-ed7150616716 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44452025-03-20T22:04:04.295ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4ff64925-7008-45b6-8056-6e3ae45a7c59 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44462025-03-20T22:04:04.296ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4ff64925-7008-45b6-8056-6e3ae45a7c59 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44472025-03-20T22:04:04.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2551 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4ff64925-7008-45b6-8056-6e3ae45a7c59 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
44482025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
44492025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
44502025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
44512025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44522025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44532025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
44542025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
44552025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44562025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
44572025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
44582025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44592025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
44602025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
44612025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
44622025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
44632025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
44642025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
44652025-03-20T22:04:05.238ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
44662025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44672025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
44682025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
44692025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
44702025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
44712025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
44722025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
44732025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
44742025-03-20T22:04:05.238ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f
44752025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
44762025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.091\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f"}
44772025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
44782025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
44792025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
44802025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
44812025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
44822025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44832025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44842025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
44852025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
44862025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
44872025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
44882025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44892025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
44902025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
44912025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44922025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
44932025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
44942025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44952025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
44962025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
44972025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
44982025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44992025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
45002025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
45012025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
45022025-03-20T22:04:05.238ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
45032025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
45042025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
45052025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
45062025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
45072025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
45082025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
45092025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
45102025-03-20T22:04:05.238ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-20 21:54:05.238554412 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
45112025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
45122025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
45132025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45142025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
45152025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
45162025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45172025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45182025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
45192025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
45202025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
45212025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
45222025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
45232025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45242025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45252025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45262025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
45272025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
45282025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
45292025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
45302025-03-20T22:04:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45312025-03-20T22:04:05.238ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58723511-3cb5-4c26-bebf-16620d9cd641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("59acd018-3b3d-4cb6-bd0f-2dfaf2b8d965.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 3789, tv_nsec: 45822872 } })
45322025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
45332025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45342025-03-20T22:04:05.239ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55400d62-c4af-483b-8223-1234d14cf0ad.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3789, tv_nsec: 45844642 } })
45352025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
45362025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45372025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
45382025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
45392025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45402025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
45412025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
45422025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
45432025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.063438ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
45442025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
45452025-03-20T22:04:05.239ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.194082ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
45462025-03-20T22:04:05.241ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
45472025-03-20T22:04:05.241ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
45482025-03-20T22:04:05.241ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
45492025-03-20T22:04:05.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
45502025-03-20T22:04:05.243ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.645034ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
45512025-03-20T22:04:05.244ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.076503ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
45522025-03-20T22:04:05.245ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45532025-03-20T22:04:05.246ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3789, tv_nsec: 53322213 } })
45542025-03-20T22:04:05.246ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
45552025-03-20T22:04:05.246ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
45562025-03-20T22:04:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
45572025-03-20T22:04:05.249ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
45582025-03-20T22:04:05.249ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45592025-03-20T22:04:05.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45602025-03-20T22:04:05.249ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45612025-03-20T22:04:05.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45622025-03-20T22:04:05.252ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45632025-03-20T22:04:05.254ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45642025-03-20T22:04:05.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45652025-03-20T22:04:05.255ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45662025-03-20T22:04:05.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45672025-03-20T22:04:05.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45682025-03-20T22:04:05.261ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45692025-03-20T22:04:05.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
45702025-03-20T22:04:05.263ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
45712025-03-20T22:04:05.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
45722025-03-20T22:04:05.269ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
45732025-03-20T22:04:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
45742025-03-20T22:04:05.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45752025-03-20T22:04:05.280ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3789, tv_nsec: 85687088 } })
45762025-03-20T22:04:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
45772025-03-20T22:04:05.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
45782025-03-20T22:04:05.282ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
45792025-03-20T22:04:05.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45802025-03-20T22:04:05.284ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45812025-03-20T22:04:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
45822025-03-20T22:04:05.286ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45832025-03-20T22:04:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45842025-03-20T22:04:05.289ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45852025-03-20T22:04:05.289ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-20 21:54:05.238554412 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
45862025-03-20T22:04:05.289ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 50.671026ms iteration = 14 status = {"expiration":"2025-03-20T21:54:05.238554412Z","pruned":[]}
45872025-03-20T22:04:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
45882025-03-20T22:04:05.292ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
45892025-03-20T22:04:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
45902025-03-20T22:04:05.295ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45912025-03-20T22:04:05.296ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45922025-03-20T22:04:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45932025-03-20T22:04:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
45942025-03-20T22:04:05.301ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45952025-03-20T22:04:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45962025-03-20T22:04:05.302ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45972025-03-20T22:04:05.304ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
45982025-03-20T22:04:05.304ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 66.593431ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
45992025-03-20T22:04:05.305ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3789, tv_nsec: 112041843 } })
46002025-03-20T22:04:05.305ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
46012025-03-20T22:04:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
46022025-03-20T22:04:05.309ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 71.156354ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
46032025-03-20T22:04:05.312ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46042025-03-20T22:04:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46052025-03-20T22:04:05.316ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46062025-03-20T22:04:05.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46072025-03-20T22:04:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46082025-03-20T22:04:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46092025-03-20T22:04:05.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fb321140-41b3-48f6-a2df-acc2a1c17f4f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
46102025-03-20T22:04:05.317ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fb321140-41b3-48f6-a2df-acc2a1c17f4f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
46112025-03-20T22:04:05.319ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.055148ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46122025-03-20T22:04:05.320ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3112 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fb321140-41b3-48f6-a2df-acc2a1c17f4f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
46132025-03-20T22:04:05.325ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 87.287384ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
46142025-03-20T22:04:05.327ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46152025-03-20T22:04:05.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46162025-03-20T22:04:05.331ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46172025-03-20T22:04:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46182025-03-20T22:04:05.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
46192025-03-20T22:04:05.331ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46202025-03-20T22:04:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46212025-03-20T22:04:05.332ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 94.053336ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
46222025-03-20T22:04:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46232025-03-20T22:04:05.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46242025-03-20T22:04:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46252025-03-20T22:04:05.334ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
46262025-03-20T22:04:05.334ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
46272025-03-20T22:04:05.334ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 95.866944ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
46282025-03-20T22:04:05.348ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46292025-03-20T22:04:05.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46302025-03-20T22:04:05.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46312025-03-20T22:04:05.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46322025-03-20T22:04:05.350ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
46332025-03-20T22:04:05.350ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
46342025-03-20T22:04:05.350ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
46352025-03-20T22:04:05.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46362025-03-20T22:04:05.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46372025-03-20T22:04:05.355ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 117.135723ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
46382025-03-20T22:04:05.356ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46392025-03-20T22:04:05.356ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46402025-03-20T22:04:05.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46412025-03-20T22:04:05.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46422025-03-20T22:04:05.359ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46432025-03-20T22:04:05.360ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
46442025-03-20T22:04:05.360ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
46452025-03-20T22:04:05.360ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46462025-03-20T22:04:05.360ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46472025-03-20T22:04:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
46482025-03-20T22:04:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
46492025-03-20T22:04:05.361ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46502025-03-20T22:04:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46512025-03-20T22:04:05.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46522025-03-20T22:04:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
46532025-03-20T22:04:05.362ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46542025-03-20T22:04:05.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46552025-03-20T22:04:05.363ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
46562025-03-20T22:04:05.363ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46572025-03-20T22:04:05.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46582025-03-20T22:04:05.364ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
46592025-03-20T22:04:05.364ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
46602025-03-20T22:04:05.364ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
46612025-03-20T22:04:05.364ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46622025-03-20T22:04:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46632025-03-20T22:04:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
46642025-03-20T22:04:05.364ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46652025-03-20T22:04:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46662025-03-20T22:04:05.365ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46672025-03-20T22:04:05.365ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46682025-03-20T22:04:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46692025-03-20T22:04:05.365ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46702025-03-20T22:04:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46712025-03-20T22:04:05.368ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46722025-03-20T22:04:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46732025-03-20T22:04:05.368ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46742025-03-20T22:04:05.369ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46752025-03-20T22:04:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46762025-03-20T22:04:05.371ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46772025-03-20T22:04:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
46782025-03-20T22:04:05.373ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46792025-03-20T22:04:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46802025-03-20T22:04:05.374ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
46812025-03-20T22:04:05.374ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46822025-03-20T22:04:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46832025-03-20T22:04:05.376ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
46842025-03-20T22:04:05.376ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
46852025-03-20T22:04:05.376ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 137.639361ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
46862025-03-20T22:04:05.385ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-20T21:51:01.993052Z, time_modified: 2025-03-20T21:51:01.993052Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }) }
46872025-03-20T22:04:05.385ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 147.390183ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]}},"default_endpoint":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
46882025-03-20T22:04:05.395ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
46892025-03-20T22:04:05.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
46902025-03-20T22:04:05.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46912025-03-20T22:04:05.400ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 161.88726ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
46922025-03-20T22:04:05.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
46932025-03-20T22:04:05.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46942025-03-20T22:04:05.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46952025-03-20T22:04:05.408ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
46962025-03-20T22:04:05.408ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
46972025-03-20T22:04:05.408ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 170.561112ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
46982025-03-20T22:04:05.409ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
46992025-03-20T22:04:05.409ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47002025-03-20T22:04:05.410ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "58eb9c03-44c0-440e-b88f-69a0d1ebf0dd", "content-length": "26", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
47012025-03-20T22:04:05.410ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47022025-03-20T22:04:05.410ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
47032025-03-20T22:04:05.410ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 171.870807ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
47042025-03-20T22:04:05.411ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47052025-03-20T22:04:05.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47062025-03-20T22:04:05.411ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
47072025-03-20T22:04:05.411ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 173.272375ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
47082025-03-20T22:04:05.413ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47092025-03-20T22:04:05.413ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 175.444883ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
47102025-03-20T22:04:05.413ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47112025-03-20T22:04:05.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47122025-03-20T22:04:05.415ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47132025-03-20T22:04:05.417ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
47142025-03-20T22:04:05.417ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
47152025-03-20T22:04:05.417ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47162025-03-20T22:04:05.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47172025-03-20T22:04:05.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
47182025-03-20T22:04:05.418ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47192025-03-20T22:04:05.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47202025-03-20T22:04:05.420ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47212025-03-20T22:04:05.420ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
47222025-03-20T22:04:05.420ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 0 version = 2
47232025-03-20T22:04:05.420ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47242025-03-20T22:04:05.420ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 182.4516ms iteration = 27 status = 0
47252025-03-20T22:04:05.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47262025-03-20T22:04:05.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
47272025-03-20T22:04:05.422ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47282025-03-20T22:04:05.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47292025-03-20T22:04:05.424ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47302025-03-20T22:04:05.426ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
47312025-03-20T22:04:05.426ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47322025-03-20T22:04:05.426ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cc522c4-7a00-4426-9fa9-fe8291b82d6f", "content-length": "26", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
47332025-03-20T22:04:05.426ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47342025-03-20T22:04:05.427ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
47352025-03-20T22:04:05.427ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47362025-03-20T22:04:05.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47372025-03-20T22:04:05.428ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
47382025-03-20T22:04:05.428ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 1 version = 2
47392025-03-20T22:04:05.428ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
47402025-03-20T22:04:05.428ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-20 21:51:07.691085 UTC dns_group = external generation = 2
47412025-03-20T22:04:05.428ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 190.198688ms iteration = 14 status = {"generation":2}
47422025-03-20T22:04:05.429ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
47432025-03-20T22:04:05.429ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
47442025-03-20T22:04:05.429ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-20 21:51:06.492803 UTC dns_group = internal generation = 1
47452025-03-20T22:04:05.429ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 191.383309ms iteration = 14 status = {"generation":1}
47462025-03-20T22:04:05.461ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
47472025-03-20T22:04:05.461ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47482025-03-20T22:04:05.461ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "796c97f7-b674-40d3-a16a-67f1fa9f0ffa", "content-length": "26", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
47492025-03-20T22:04:05.461ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47502025-03-20T22:04:05.462ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47512025-03-20T22:04:05.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47522025-03-20T22:04:05.463ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
47532025-03-20T22:04:05.463ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 225.545163ms iteration = 30 status = {"physical_disks_added":0}
47542025-03-20T22:04:05.518ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
47552025-03-20T22:04:05.521ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
47562025-03-20T22:04:05.521ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 283.634726ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
47572025-03-20T22:04:05.569ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
47582025-03-20T22:04:05.588ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 350.272298ms iteration = 27 status = {}
47592025-03-20T22:04:05.589ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
47602025-03-20T22:04:05.590ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ab13cda-40e5-4807-9bb8-3cd0864611ed", "content-length": "2", "date": "Thu, 20 Mar 2025 22:04:04 GMT"} })
47612025-03-20T22:04:05.596ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
47622025-03-20T22:04:05.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
47632025-03-20T22:04:05.607ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
47642025-03-20T22:04:05.607ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47652025-03-20T22:04:05.608ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3939040-94e5-4f84-9198-d4bb57ecbfff", "content-length": "372", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
47662025-03-20T22:04:05.608ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47672025-03-20T22:04:05.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47682025-03-20T22:04:05.609ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47692025-03-20T22:04:05.611ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 373.049218ms iteration = 27 status = null
47702025-03-20T22:04:05.611ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47712025-03-20T22:04:05.611ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47722025-03-20T22:04:05.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47732025-03-20T22:04:05.613ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47742025-03-20T22:04:05.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
47752025-03-20T22:04:05.617ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47762025-03-20T22:04:05.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47772025-03-20T22:04:05.670ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
47782025-03-20T22:04:05.671ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
47792025-03-20T22:04:05.671ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "103555f1-a423-4f31-b0dc-6c5ebd776c1b", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
47802025-03-20T22:04:05.671ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47812025-03-20T22:04:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47822025-03-20T22:04:05.673ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47832025-03-20T22:04:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47842025-03-20T22:04:05.676ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47852025-03-20T22:04:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47862025-03-20T22:04:05.681ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
47872025-03-20T22:04:05.681ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47882025-03-20T22:04:05.681ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8e959af2-6de4-4e99-81a0-3481d8df7c42", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
47892025-03-20T22:04:05.683ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 445.789243ms iteration = 28 status = {}
47902025-03-20T22:04:05.732ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
47912025-03-20T22:04:05.733ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "25762b83-8344-4dff-bf01-6067a49545f9", "content-length": "691", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
47922025-03-20T22:04:05.733ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
47932025-03-20T22:04:05.733ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
47942025-03-20T22:04:05.733ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
47952025-03-20T22:04:05.735ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 497.791274ms iteration = 28 status = {}
47962025-03-20T22:04:05.762ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
47972025-03-20T22:04:05.762ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b43743d8-8c12-4b70-9081-97127a5ecc61", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
47982025-03-20T22:04:05.762ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
47992025-03-20T22:04:05.763ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
48002025-03-20T22:04:05.764ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f7aa5f58-0c68-44a0-990d-5aa11a989864", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48012025-03-20T22:04:05.764ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48022025-03-20T22:04:05.766ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48032025-03-20T22:04:05.766ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 527.877899ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
48042025-03-20T22:04:05.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48052025-03-20T22:04:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48062025-03-20T22:04:05.792ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
48072025-03-20T22:04:05.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48082025-03-20T22:04:05.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48092025-03-20T22:04:05.799ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
48102025-03-20T22:04:05.799ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "434b495a-2b87-49de-816e-e2df7b290043", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
48112025-03-20T22:04:05.799ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
48122025-03-20T22:04:05.800ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
48132025-03-20T22:04:05.801ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "53a2e5b0-6493-4148-a15a-8fc4aadc834d", "content-length": "134", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
48142025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
48152025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
48162025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
48172025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
48182025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48192025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48202025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48212025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48222025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
48232025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
48242025-03-20T22:04:05.801ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
48252025-03-20T22:04:05.801ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48262025-03-20T22:04:05.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48272025-03-20T22:04:05.802ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "2c6371ea-1a66-4f77-837c-9806d272608b", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
48282025-03-20T22:04:05.802ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
48292025-03-20T22:04:05.804ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
48302025-03-20T22:04:05.804ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "8787e1a1-9392-4d8f-a7b9-f09aa7a6b212", "content-length": "2", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
48312025-03-20T22:04:05.804ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
48322025-03-20T22:04:05.804ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "61141dd4-2408-47e4-b83b-18359fea9da0", "content-length": "52", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
48332025-03-20T22:04:05.804ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
48342025-03-20T22:04:05.804ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
48352025-03-20T22:04:05.804ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "95513cc7-3a46-44cd-9e67-f168bb1c4c07", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
48362025-03-20T22:04:05.804ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
48372025-03-20T22:04:05.806ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
48382025-03-20T22:04:05.806ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 568.363772ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
48392025-03-20T22:04:05.998ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6a4a9684-9e48-47ca-8222-6e821c46d3de", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
48402025-03-20T22:04:05.998ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
48412025-03-20T22:04:05.998ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
48422025-03-20T22:04:05.999ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "32dda8b9-ac21-4e4a-a5b4-d00bf90520aa", "content-length": "682", "date": "Thu, 20 Mar 2025 22:04:05 GMT"} })
48432025-03-20T22:04:05.999ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48442025-03-20T22:04:05.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48452025-03-20T22:04:06.002ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48462025-03-20T22:04:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48472025-03-20T22:04:06.006ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48482025-03-20T22:04:06.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48492025-03-20T22:04:06.009ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48502025-03-20T22:04:06.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48512025-03-20T22:04:06.016ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 778.073151ms iteration = 27 status = {}
48522025-03-20T22:04:06.352ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9d3e3291-041c-45b4-860e-f3217c3285da roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48532025-03-20T22:04:06.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9d3e3291-041c-45b4-860e-f3217c3285da resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48542025-03-20T22:04:06.354ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2177 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9d3e3291-041c-45b4-860e-f3217c3285da response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48552025-03-20T22:04:07.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 556 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = d3df0a4b-b707-451b-a76c-53c918a5ae4c response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
48562025-03-20T22:04:07.374ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a1179fd9-8ce9-420f-9689-c3329f9ca8bc roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48572025-03-20T22:04:07.374ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a1179fd9-8ce9-420f-9689-c3329f9ca8bc resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48582025-03-20T22:04:07.376ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a1179fd9-8ce9-420f-9689-c3329f9ca8bc response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48592025-03-20T22:04:08.396ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 146acb3e-b253-4acb-a849-e6f19baf731f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48602025-03-20T22:04:08.397ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 146acb3e-b253-4acb-a849-e6f19baf731f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48612025-03-20T22:04:08.398ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2060 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 146acb3e-b253-4acb-a849-e6f19baf731f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48622025-03-20T22:04:09.418ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0759785-dd7c-4c57-84f9-633b9c7aee7a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48632025-03-20T22:04:09.418ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0759785-dd7c-4c57-84f9-633b9c7aee7a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48642025-03-20T22:04:09.420ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2273 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b0759785-dd7c-4c57-84f9-633b9c7aee7a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48652025-03-20T22:04:10.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cab6e1eb-cac4-4e01-97a7-912a295aa68e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48662025-03-20T22:04:10.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cab6e1eb-cac4-4e01-97a7-912a295aa68e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48672025-03-20T22:04:10.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2500 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = cab6e1eb-cac4-4e01-97a7-912a295aa68e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48682025-03-20T22:04:11.462ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 281b186b-feca-45df-b898-01985957a45c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48692025-03-20T22:04:11.462ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 281b186b-feca-45df-b898-01985957a45c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48702025-03-20T22:04:11.464ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 281b186b-feca-45df-b898-01985957a45c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48712025-03-20T22:04:12.484ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 55ff633e-c75d-438d-969b-a0a3374ecdd6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48722025-03-20T22:04:12.484ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 55ff633e-c75d-438d-969b-a0a3374ecdd6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48732025-03-20T22:04:12.486ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2202 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 55ff633e-c75d-438d-969b-a0a3374ecdd6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48742025-03-20T22:04:13.505ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 35103ae1-2cd8-438a-8dc8-70003d8274cd roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48752025-03-20T22:04:13.506ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 35103ae1-2cd8-438a-8dc8-70003d8274cd resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48762025-03-20T22:04:13.507ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2068 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 35103ae1-2cd8-438a-8dc8-70003d8274cd response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48772025-03-20T22:04:14.527ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 945a014b-7c4f-4e1b-a772-ceebf4419089 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48782025-03-20T22:04:14.527ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 945a014b-7c4f-4e1b-a772-ceebf4419089 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48792025-03-20T22:04:14.529ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2788 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 945a014b-7c4f-4e1b-a772-ceebf4419089 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
48802025-03-20T22:04:15.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
48812025-03-20T22:04:15.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48822025-03-20T22:04:15.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
48832025-03-20T22:04:15.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48842025-03-20T22:04:15.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48852025-03-20T22:04:15.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48862025-03-20T22:04:15.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48872025-03-20T22:04:15.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48882025-03-20T22:04:15.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48892025-03-20T22:04:15.240ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.513765ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
48902025-03-20T22:04:15.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48912025-03-20T22:04:15.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48922025-03-20T22:04:15.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48932025-03-20T22:04:15.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48942025-03-20T22:04:15.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48952025-03-20T22:04:15.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48962025-03-20T22:04:15.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
48972025-03-20T22:04:15.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48982025-03-20T22:04:15.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48992025-03-20T22:04:15.254ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
49002025-03-20T22:04:15.254ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.759328ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
49012025-03-20T22:04:15.549ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 85ff74c8-48ae-4524-b4d7-9b63f87297bc roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49022025-03-20T22:04:15.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 85ff74c8-48ae-4524-b4d7-9b63f87297bc resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49032025-03-20T22:04:15.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1949 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 85ff74c8-48ae-4524-b4d7-9b63f87297bc response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49042025-03-20T22:04:16.571ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5f0366d9-abdd-47e4-b4fc-8f569c402816 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49052025-03-20T22:04:16.572ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5f0366d9-abdd-47e4-b4fc-8f569c402816 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49062025-03-20T22:04:16.574ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2793 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 5f0366d9-abdd-47e4-b4fc-8f569c402816 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49072025-03-20T22:04:17.246ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = f2348ddc-1d40-4876-b216-bf25eb3b94ef roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
49082025-03-20T22:04:17.246ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = f2348ddc-1d40-4876-b216-bf25eb3b94ef resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
49092025-03-20T22:04:17.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3448 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = f2348ddc-1d40-4876-b216-bf25eb3b94ef response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
49102025-03-20T22:04:17.250ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 73c5a86e-24ae-4a68-9b5f-4a2227ad3694 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
49112025-03-20T22:04:17.250ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 73c5a86e-24ae-4a68-9b5f-4a2227ad3694 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
49122025-03-20T22:04:17.252ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2160 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 73c5a86e-24ae-4a68-9b5f-4a2227ad3694 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
49132025-03-20T22:04:17.252ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 565 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = c5db887b-695d-4e32-8034-702e485d3218 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
49142025-03-20T22:04:17.593ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 473f5c6d-9143-47f7-8c7d-2f3c7237de94 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49152025-03-20T22:04:17.593ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 473f5c6d-9143-47f7-8c7d-2f3c7237de94 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49162025-03-20T22:04:17.595ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2628 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 473f5c6d-9143-47f7-8c7d-2f3c7237de94 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49172025-03-20T22:04:18.615ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 82db0091-46ac-43d0-aeae-6a5cc61ff178 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49182025-03-20T22:04:18.615ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 82db0091-46ac-43d0-aeae-6a5cc61ff178 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49192025-03-20T22:04:18.617ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2317 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 82db0091-46ac-43d0-aeae-6a5cc61ff178 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49202025-03-20T22:04:19.637ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d2158e86-0b0b-4e63-bece-ea72014dbf87 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49212025-03-20T22:04:19.637ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d2158e86-0b0b-4e63-bece-ea72014dbf87 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49222025-03-20T22:04:19.639ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2117 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d2158e86-0b0b-4e63-bece-ea72014dbf87 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49232025-03-20T22:04:20.659ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fddcc571-4a7e-409c-a7fd-db370197bc64 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49242025-03-20T22:04:20.659ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fddcc571-4a7e-409c-a7fd-db370197bc64 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49252025-03-20T22:04:20.661ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2080 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fddcc571-4a7e-409c-a7fd-db370197bc64 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49262025-03-20T22:04:21.680ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c75442cb-9d6f-408d-b743-ce22efcb292d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49272025-03-20T22:04:21.680ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c75442cb-9d6f-408d-b743-ce22efcb292d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49282025-03-20T22:04:21.682ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2469 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c75442cb-9d6f-408d-b743-ce22efcb292d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49292025-03-20T22:04:22.701ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 373adb8b-5139-4a98-9242-97dfe3120465 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49302025-03-20T22:04:22.702ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 373adb8b-5139-4a98-9242-97dfe3120465 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49312025-03-20T22:04:22.704ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2454 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 373adb8b-5139-4a98-9242-97dfe3120465 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49322025-03-20T22:04:23.722ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 37be87d2-ece5-4a1b-a24f-a2e2c0b404c9 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49332025-03-20T22:04:23.723ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 37be87d2-ece5-4a1b-a24f-a2e2c0b404c9 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49342025-03-20T22:04:23.725ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2491 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 37be87d2-ece5-4a1b-a24f-a2e2c0b404c9 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49352025-03-20T22:04:24.746ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 57fc99be-435a-40e0-9aff-75b050ae6141 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49362025-03-20T22:04:24.746ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 57fc99be-435a-40e0-9aff-75b050ae6141 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49372025-03-20T22:04:24.748ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2616 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 57fc99be-435a-40e0-9aff-75b050ae6141 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49382025-03-20T22:04:25.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
49392025-03-20T22:04:25.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
49402025-03-20T22:04:25.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49412025-03-20T22:04:25.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49422025-03-20T22:04:25.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49432025-03-20T22:04:25.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49442025-03-20T22:04:25.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49452025-03-20T22:04:25.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
49462025-03-20T22:04:25.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49472025-03-20T22:04:25.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49482025-03-20T22:04:25.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49492025-03-20T22:04:25.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49502025-03-20T22:04:25.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49512025-03-20T22:04:25.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
49522025-03-20T22:04:25.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49532025-03-20T22:04:25.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49542025-03-20T22:04:25.256ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
49552025-03-20T22:04:25.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.762151ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
49562025-03-20T22:04:25.268ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49572025-03-20T22:04:25.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49582025-03-20T22:04:25.270ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.755257ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49592025-03-20T22:04:25.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b359e6d2-7f3c-4c20-ac9f-8cf449d061be roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49602025-03-20T22:04:25.767ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b359e6d2-7f3c-4c20-ac9f-8cf449d061be resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49612025-03-20T22:04:25.769ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2200 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b359e6d2-7f3c-4c20-ac9f-8cf449d061be response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49622025-03-20T22:04:26.790ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee047dce-a8be-47b3-9f99-f02bbde019ac roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49632025-03-20T22:04:26.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee047dce-a8be-47b3-9f99-f02bbde019ac resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49642025-03-20T22:04:26.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2582 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ee047dce-a8be-47b3-9f99-f02bbde019ac response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49652025-03-20T22:04:27.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 866 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 31c97ff4-b1b2-4129-a2dc-2f4d7ab3909c response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
49662025-03-20T22:04:27.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bd5472ef-f8ce-41df-8753-b87cbd3522a6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49672025-03-20T22:04:27.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bd5472ef-f8ce-41df-8753-b87cbd3522a6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49682025-03-20T22:04:27.813ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2254 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bd5472ef-f8ce-41df-8753-b87cbd3522a6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49692025-03-20T22:04:28.834ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d7af1cde-4e3b-403c-bb7a-fdb0ad10829c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49702025-03-20T22:04:28.834ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d7af1cde-4e3b-403c-bb7a-fdb0ad10829c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49712025-03-20T22:04:28.836ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2377 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = d7af1cde-4e3b-403c-bb7a-fdb0ad10829c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49722025-03-20T22:04:29.856ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 23701023-e133-46fe-92e6-03152cb8229d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49732025-03-20T22:04:29.857ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 23701023-e133-46fe-92e6-03152cb8229d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49742025-03-20T22:04:29.858ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1997 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 23701023-e133-46fe-92e6-03152cb8229d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49752025-03-20T22:04:30.877ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 312b480b-4bc6-4c97-a0f3-4feff86f554e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49762025-03-20T22:04:30.878ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 312b480b-4bc6-4c97-a0f3-4feff86f554e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49772025-03-20T22:04:30.880ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2725 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 312b480b-4bc6-4c97-a0f3-4feff86f554e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49782025-03-20T22:04:31.899ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = df3e65f8-6129-480a-b47f-fdc07f31d93a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49792025-03-20T22:04:31.899ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = df3e65f8-6129-480a-b47f-fdc07f31d93a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49802025-03-20T22:04:31.901ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2749 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = df3e65f8-6129-480a-b47f-fdc07f31d93a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49812025-03-20T22:04:32.921ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 66771905-99b1-4efb-a105-295480153089 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49822025-03-20T22:04:32.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 66771905-99b1-4efb-a105-295480153089 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49832025-03-20T22:04:32.924ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2704 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 66771905-99b1-4efb-a105-295480153089 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49842025-03-20T22:04:33.943ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 06d90c92-613f-4d6f-b71d-1a1f65cec7e5 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49852025-03-20T22:04:33.944ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 06d90c92-613f-4d6f-b71d-1a1f65cec7e5 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49862025-03-20T22:04:33.946ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2492 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 06d90c92-613f-4d6f-b71d-1a1f65cec7e5 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49872025-03-20T22:04:34.964ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e345bd11-0654-4e7e-a04c-f11299e93eb7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49882025-03-20T22:04:34.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e345bd11-0654-4e7e-a04c-f11299e93eb7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49892025-03-20T22:04:34.966ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2222 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e345bd11-0654-4e7e-a04c-f11299e93eb7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
49902025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
49912025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
49922025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
49932025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49942025-03-20T22:04:35.237ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
49952025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
49962025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
49972025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49982025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
49992025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
50002025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
50012025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
50022025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
50032025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
50042025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50052025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
50062025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50072025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
50082025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50092025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
50102025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
50112025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
50122025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
50132025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
50142025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
50152025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
50162025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50172025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
50182025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
50192025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
50202025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
50212025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
50222025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50232025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
50242025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
50252025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
50262025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50272025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
50282025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
50292025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50302025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50312025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
50322025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50332025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50342025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50352025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
50362025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50372025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
50382025-03-20T22:04:35.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
50392025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
50402025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
50412025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
50422025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
50432025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50442025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
50452025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
50462025-03-20T22:04:35.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50472025-03-20T22:04:35.242ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50482025-03-20T22:04:35.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50492025-03-20T22:04:35.244ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.775381ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
50502025-03-20T22:04:35.244ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
50512025-03-20T22:04:35.244ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50522025-03-20T22:04:35.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50532025-03-20T22:04:35.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50542025-03-20T22:04:35.244ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50552025-03-20T22:04:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50562025-03-20T22:04:35.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50572025-03-20T22:04:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50582025-03-20T22:04:35.246ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50592025-03-20T22:04:35.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50602025-03-20T22:04:35.247ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
50612025-03-20T22:04:35.247ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50622025-03-20T22:04:35.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50632025-03-20T22:04:35.247ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
50642025-03-20T22:04:35.247ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50652025-03-20T22:04:35.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50662025-03-20T22:04:35.248ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.177039ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
50672025-03-20T22:04:35.249ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.299759ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
50682025-03-20T22:04:35.250ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50692025-03-20T22:04:35.251ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3819, tv_nsec: 58367427 } })
50702025-03-20T22:04:35.251ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
50712025-03-20T22:04:35.251ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50722025-03-20T22:04:35.252ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3819, tv_nsec: 59220720 } })
50732025-03-20T22:04:35.252ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
50742025-03-20T22:04:35.253ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
50752025-03-20T22:04:35.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
50762025-03-20T22:04:35.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
50772025-03-20T22:04:35.254ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
50782025-03-20T22:04:35.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
50792025-03-20T22:04:35.256ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
50802025-03-20T22:04:35.256ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50812025-03-20T22:04:35.256ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
50822025-03-20T22:04:35.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
50832025-03-20T22:04:35.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
50842025-03-20T22:04:35.256ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50852025-03-20T22:04:35.257ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50862025-03-20T22:04:35.258ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 21.263849ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
50872025-03-20T22:04:35.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50882025-03-20T22:04:35.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
50892025-03-20T22:04:35.259ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50902025-03-20T22:04:35.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
50912025-03-20T22:04:35.260ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
50922025-03-20T22:04:35.260ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50932025-03-20T22:04:35.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50942025-03-20T22:04:35.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
50952025-03-20T22:04:35.260ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
50962025-03-20T22:04:35.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
50972025-03-20T22:04:35.264ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3819, tv_nsec: 71626492 } })
50982025-03-20T22:04:35.264ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
50992025-03-20T22:04:35.264ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
51002025-03-20T22:04:35.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51012025-03-20T22:04:35.268ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51022025-03-20T22:04:35.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51032025-03-20T22:04:35.271ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
51042025-03-20T22:04:35.271ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 33.888877ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
51052025-03-20T22:04:35.292ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51062025-03-20T22:04:35.292ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
51072025-03-20T22:04:35.292ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51082025-03-20T22:04:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51092025-03-20T22:04:35.300ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51102025-03-20T22:04:35.300ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 63.284993ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
51112025-03-20T22:04:35.300ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.270423ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51122025-03-20T22:04:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51132025-03-20T22:04:35.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
51142025-03-20T22:04:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
51152025-03-20T22:04:35.304ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
51162025-03-20T22:04:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
51172025-03-20T22:04:35.305ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 68.560054ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
51182025-03-20T22:04:35.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51192025-03-20T22:04:35.306ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51202025-03-20T22:04:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51212025-03-20T22:04:35.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51222025-03-20T22:04:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51232025-03-20T22:04:35.310ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51242025-03-20T22:04:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
51252025-03-20T22:04:35.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51262025-03-20T22:04:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51272025-03-20T22:04:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
51282025-03-20T22:04:35.313ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
51292025-03-20T22:04:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
51302025-03-20T22:04:35.315ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 78.368716ms iteration = 28 status = 0
51312025-03-20T22:04:35.321ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
51322025-03-20T22:04:35.321ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.819708ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
51332025-03-20T22:04:35.323ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
51342025-03-20T22:04:35.323ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51352025-03-20T22:04:35.323ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5311cae0-2410-475b-9684-ac2648f09dcc", "content-length": "26", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
51362025-03-20T22:04:35.323ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51372025-03-20T22:04:35.324ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51382025-03-20T22:04:35.325ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
51392025-03-20T22:04:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51402025-03-20T22:04:35.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51412025-03-20T22:04:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51422025-03-20T22:04:35.330ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51432025-03-20T22:04:35.330ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
51442025-03-20T22:04:35.330ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
51452025-03-20T22:04:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
51462025-03-20T22:04:35.333ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
51472025-03-20T22:04:35.333ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51482025-03-20T22:04:35.333ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "620bb091-c18e-4123-aab9-7514fcc9bdbd", "content-length": "26", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
51492025-03-20T22:04:35.333ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51502025-03-20T22:04:35.336ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
51512025-03-20T22:04:35.339ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 101.825224ms iteration = 31 status = {"physical_disks_added":0}
51522025-03-20T22:04:35.341ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 103.989142ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
51532025-03-20T22:04:35.345ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51542025-03-20T22:04:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
51552025-03-20T22:04:35.347ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
51562025-03-20T22:04:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
51572025-03-20T22:04:35.349ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
51582025-03-20T22:04:35.349ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 112.418487ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
51592025-03-20T22:04:35.374ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
51602025-03-20T22:04:35.374ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
51612025-03-20T22:04:35.374ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 137.22051ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
51622025-03-20T22:04:35.392ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51632025-03-20T22:04:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51642025-03-20T22:04:35.427ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
51652025-03-20T22:04:35.427ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
51662025-03-20T22:04:35.428ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bf0d23b-e8f9-48a4-8e17-4a832ab861d7", "content-length": "26", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
51672025-03-20T22:04:35.428ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
51682025-03-20T22:04:35.430ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
51692025-03-20T22:04:35.432ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 194.856412ms iteration = 28 status = {}
51702025-03-20T22:04:35.570ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
51712025-03-20T22:04:35.570ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "6930a980-55ef-42f9-9aea-1ba8b7c973d8", "content-length": "2", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
51722025-03-20T22:04:35.572ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
51732025-03-20T22:04:35.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
51742025-03-20T22:04:35.574ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
51752025-03-20T22:04:35.574ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
51762025-03-20T22:04:35.577ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 340.386574ms iteration = 28 status = null
51772025-03-20T22:04:35.613ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51782025-03-20T22:04:35.615ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
51792025-03-20T22:04:35.615ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51802025-03-20T22:04:35.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51812025-03-20T22:04:35.617ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51822025-03-20T22:04:35.619ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
51832025-03-20T22:04:35.619ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
51842025-03-20T22:04:35.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
51852025-03-20T22:04:35.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51862025-03-20T22:04:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51872025-03-20T22:04:35.646ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
51882025-03-20T22:04:35.646ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51892025-03-20T22:04:35.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51902025-03-20T22:04:35.656ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
51912025-03-20T22:04:35.657ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec9bfe90-7070-4427-8e9d-936d20122c65", "content-length": "134", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
51922025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
51932025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
51942025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
51952025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
51962025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51972025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51982025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
51992025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
52002025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
52012025-03-20T22:04:35.657ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
52022025-03-20T22:04:35.658ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
52032025-03-20T22:04:35.658ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52042025-03-20T22:04:35.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52052025-03-20T22:04:35.664ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
52062025-03-20T22:04:35.666ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
52072025-03-20T22:04:35.666ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bf277f8-7076-4003-8b07-b378303d08a1", "content-length": "2", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52082025-03-20T22:04:35.666ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
52092025-03-20T22:04:35.666ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "8da37c06-663c-4727-871c-1f136ed23119", "content-length": "52", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52102025-03-20T22:04:35.666ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
52112025-03-20T22:04:35.666ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
52122025-03-20T22:04:35.677ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "eab07d7d-4025-40d4-b62f-a1df15577bd5", "content-length": "372", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52132025-03-20T22:04:35.677ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52142025-03-20T22:04:35.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52152025-03-20T22:04:35.683ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "01917f65-4a36-44e9-ab7d-6531a510ef96", "content-length": "691", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52162025-03-20T22:04:35.683ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
52172025-03-20T22:04:35.683ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
52182025-03-20T22:04:35.683ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
52192025-03-20T22:04:35.684ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 447.435957ms iteration = 29 status = {}
52202025-03-20T22:04:35.722ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
52212025-03-20T22:04:35.722ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
52222025-03-20T22:04:35.725ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6a03fac8-f5da-4978-86ce-bfc8bb616e2e", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52232025-03-20T22:04:35.725ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52242025-03-20T22:04:35.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52252025-03-20T22:04:35.728ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52262025-03-20T22:04:35.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52272025-03-20T22:04:35.731ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52282025-03-20T22:04:35.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52292025-03-20T22:04:35.735ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
52302025-03-20T22:04:35.735ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
52312025-03-20T22:04:35.739ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c61fc55f-151b-492f-a29b-a71a22d5e0bf", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52322025-03-20T22:04:35.741ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 504.043081ms iteration = 29 status = {}
52332025-03-20T22:04:35.784ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5359f112-dee5-4fd3-8aa7-de684f9c4cbc", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52342025-03-20T22:04:35.784ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
52352025-03-20T22:04:35.784ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
52362025-03-20T22:04:35.784ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "40cefdb9-6c50-459f-9497-fc3dacf241f3", "content-length": "682", "date": "Thu, 20 Mar 2025 22:04:35 GMT"} })
52372025-03-20T22:04:35.784ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52382025-03-20T22:04:35.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52392025-03-20T22:04:35.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52402025-03-20T22:04:35.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52412025-03-20T22:04:35.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52422025-03-20T22:04:35.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52432025-03-20T22:04:35.794ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52442025-03-20T22:04:35.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52452025-03-20T22:04:35.800ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 563.600335ms iteration = 28 status = {}
52462025-03-20T22:04:35.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bdd18084-5c1b-4a4e-97e4-114f33d7d064 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52472025-03-20T22:04:35.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bdd18084-5c1b-4a4e-97e4-114f33d7d064 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52482025-03-20T22:04:35.988ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2136 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = bdd18084-5c1b-4a4e-97e4-114f33d7d064 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52492025-03-20T22:04:36.291ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
52502025-03-20T22:04:36.291ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
52512025-03-20T22:04:36.321ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
52522025-03-20T22:04:36.321ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:44286
52532025-03-20T22:04:36.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44286 req_id = 8f83c49b-6ee0-4e88-8005-b5d55a546f44 roles = RoleSet { roles: {} } uri = /metrics/producers
52542025-03-20T22:04:36.322ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44286 req_id = 8f83c49b-6ee0-4e88-8005-b5d55a546f44 resource = Database result = Ok(()) uri = /metrics/producers
52552025-03-20T22:04:36.357ZINFOnexus: assigned collector to new producer collector_id = c75d8631-e01d-47db-98de-f38076ccc33e file = nexus/src/app/oximeter.rs:76 producer_id = 8602d1a8-380a-4598-9f5c-9268e05f383a
52562025-03-20T22:04:36.357ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 35084 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44286 req_id = 8f83c49b-6ee0-4e88-8005-b5d55a546f44 response_code = 201 uri = /metrics/producers
52572025-03-20T22:04:36.357ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "8f83c49b-6ee0-4e88-8005-b5d55a546f44", "content-length": "41", "date": "Thu, 20 Mar 2025 22:04:36 GMT"} })
52582025-03-20T22:04:36.358ZDEBGnexus (producer-registration-task): registered with nexus successfully
52592025-03-20T22:04:36.358ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
52602025-03-20T22:04:37.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae3a9608-4838-4940-be19-f14b32982c99 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52612025-03-20T22:04:37.007ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae3a9608-4838-4940-be19-f14b32982c99 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52622025-03-20T22:04:37.008ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2199 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ae3a9608-4838-4940-be19-f14b32982c99 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52632025-03-20T22:04:37.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 615 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = ad8e3623-a6c4-466b-bbac-f7d255b1a186 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
52642025-03-20T22:04:38.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8226cc9c-9a87-4e19-97d4-bbeb9e6b745b roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52652025-03-20T22:04:38.029ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8226cc9c-9a87-4e19-97d4-bbeb9e6b745b resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52662025-03-20T22:04:38.031ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 8226cc9c-9a87-4e19-97d4-bbeb9e6b745b response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52672025-03-20T22:04:39.050ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e02cd477-b851-4efa-84eb-bd69380026d0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52682025-03-20T22:04:39.051ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e02cd477-b851-4efa-84eb-bd69380026d0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52692025-03-20T22:04:39.053ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2600 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = e02cd477-b851-4efa-84eb-bd69380026d0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52702025-03-20T22:04:40.073ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 873d75b4-a82d-4527-a652-ac4ca61bb08a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52712025-03-20T22:04:40.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 873d75b4-a82d-4527-a652-ac4ca61bb08a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52722025-03-20T22:04:40.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2305 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 873d75b4-a82d-4527-a652-ac4ca61bb08a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52732025-03-20T22:04:41.096ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 501aeb2c-89cd-4e46-b2a3-acb854da0d14 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52742025-03-20T22:04:41.096ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 501aeb2c-89cd-4e46-b2a3-acb854da0d14 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52752025-03-20T22:04:41.098ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2663 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 501aeb2c-89cd-4e46-b2a3-acb854da0d14 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52762025-03-20T22:04:42.119ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 575c42ed-ebb5-4d32-b898-588b779217f0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52772025-03-20T22:04:42.119ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 575c42ed-ebb5-4d32-b898-588b779217f0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52782025-03-20T22:04:42.121ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2100 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 575c42ed-ebb5-4d32-b898-588b779217f0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52792025-03-20T22:04:43.141ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 655ce0b5-a027-4e7c-9072-06c29f4c3fc6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52802025-03-20T22:04:43.142ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 655ce0b5-a027-4e7c-9072-06c29f4c3fc6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52812025-03-20T22:04:43.144ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2298 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 655ce0b5-a027-4e7c-9072-06c29f4c3fc6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52822025-03-20T22:04:44.164ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b51f32ba-8661-4b0b-beb0-a1d14ab18749 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52832025-03-20T22:04:44.165ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b51f32ba-8661-4b0b-beb0-a1d14ab18749 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52842025-03-20T22:04:44.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2367 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = b51f32ba-8661-4b0b-beb0-a1d14ab18749 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52852025-03-20T22:04:45.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fafc50c5-b250-4203-9c58-2d9f2f8d1c2d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52862025-03-20T22:04:45.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fafc50c5-b250-4203-9c58-2d9f2f8d1c2d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52872025-03-20T22:04:45.189ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3075 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fafc50c5-b250-4203-9c58-2d9f2f8d1c2d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
52882025-03-20T22:04:45.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
52892025-03-20T22:04:45.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
52902025-03-20T22:04:45.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52912025-03-20T22:04:45.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52922025-03-20T22:04:45.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52932025-03-20T22:04:45.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52942025-03-20T22:04:45.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52952025-03-20T22:04:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
52962025-03-20T22:04:45.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52972025-03-20T22:04:45.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52982025-03-20T22:04:45.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52992025-03-20T22:04:45.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53002025-03-20T22:04:45.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53012025-03-20T22:04:45.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53022025-03-20T22:04:45.278ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.058428ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53032025-03-20T22:04:45.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53042025-03-20T22:04:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
53052025-03-20T22:04:45.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53062025-03-20T22:04:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53072025-03-20T22:04:45.295ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
53082025-03-20T22:04:45.295ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.625571ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
53092025-03-20T22:04:46.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 36bd39e0-8da6-4b29-ab3b-ce6be00827e2 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53102025-03-20T22:04:46.209ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 36bd39e0-8da6-4b29-ab3b-ce6be00827e2 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53112025-03-20T22:04:46.211ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2426 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 36bd39e0-8da6-4b29-ab3b-ce6be00827e2 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53122025-03-20T22:04:47.231ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 031afcf4-8cbe-4e88-8708-50d66adb5e0d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53132025-03-20T22:04:47.231ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 031afcf4-8cbe-4e88-8708-50d66adb5e0d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53142025-03-20T22:04:47.233ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2489 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 031afcf4-8cbe-4e88-8708-50d66adb5e0d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53152025-03-20T22:04:47.246ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 13de7cc2-6286-45c2-a50e-bbcd69485c0f roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
53162025-03-20T22:04:47.246ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 13de7cc2-6286-45c2-a50e-bbcd69485c0f resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
53172025-03-20T22:04:47.248ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3054 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 13de7cc2-6286-45c2-a50e-bbcd69485c0f response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?sort_by=id_ascending
53182025-03-20T22:04:47.249ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 808904d4-4c2c-4b23-b74e-3528238d9096 roles = RoleSet { roles: {} } uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
53192025-03-20T22:04:47.249ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 808904d4-4c2c-4b23-b74e-3528238d9096 resource = Database result = Ok(()) uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
53202025-03-20T22:04:47.251ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2552 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42088 req_id = 808904d4-4c2c-4b23-b74e-3528238d9096 response_code = 200 uri = /metrics/collectors/c75d8631-e01d-47db-98de-f38076ccc33e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZjJmNjA1Mi1lZjZiLTQwZGYtYjBkNS1lMTg5MTZhZTViMmIifX0%3D
53212025-03-20T22:04:47.252ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 502 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = b28c06ac-8ccd-4691-a312-6a7f944c2334 response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
53222025-03-20T22:04:48.254ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 43c6f000-b58f-44e7-8f74-c53a7df3dda0 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53232025-03-20T22:04:48.254ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 43c6f000-b58f-44e7-8f74-c53a7df3dda0 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53242025-03-20T22:04:48.256ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2188 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 43c6f000-b58f-44e7-8f74-c53a7df3dda0 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53252025-03-20T22:04:49.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 63bbf533-2262-4319-978b-18738af21b5a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53262025-03-20T22:04:49.278ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 63bbf533-2262-4319-978b-18738af21b5a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53272025-03-20T22:04:49.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2069 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 63bbf533-2262-4319-978b-18738af21b5a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53282025-03-20T22:04:50.301ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a5aa7819-d9df-406b-b91c-214e9995db25 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53292025-03-20T22:04:50.301ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a5aa7819-d9df-406b-b91c-214e9995db25 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53302025-03-20T22:04:50.303ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2151 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a5aa7819-d9df-406b-b91c-214e9995db25 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53312025-03-20T22:04:51.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9af6b570-f02c-41b2-8e8c-e10e67942037 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53322025-03-20T22:04:51.323ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9af6b570-f02c-41b2-8e8c-e10e67942037 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53332025-03-20T22:04:51.325ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2365 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 9af6b570-f02c-41b2-8e8c-e10e67942037 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53342025-03-20T22:04:52.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1e98b81f-1dbd-4c08-aec8-ede0e8d2f2be roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53352025-03-20T22:04:52.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1e98b81f-1dbd-4c08-aec8-ede0e8d2f2be resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53362025-03-20T22:04:52.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2509 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 1e98b81f-1dbd-4c08-aec8-ede0e8d2f2be response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53372025-03-20T22:04:53.368ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0fbc1e77-2b76-417b-9c9b-b9fabe94fd0f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53382025-03-20T22:04:53.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0fbc1e77-2b76-417b-9c9b-b9fabe94fd0f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53392025-03-20T22:04:53.370ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0fbc1e77-2b76-417b-9c9b-b9fabe94fd0f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53402025-03-20T22:04:54.390ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c6a08e15-c784-4434-a8b3-e5bc3bc3fb7d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53412025-03-20T22:04:54.390ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c6a08e15-c784-4434-a8b3-e5bc3bc3fb7d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53422025-03-20T22:04:54.392ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2257 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = c6a08e15-c784-4434-a8b3-e5bc3bc3fb7d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53432025-03-20T22:04:55.236ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
53442025-03-20T22:04:55.236ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53452025-03-20T22:04:55.236ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
53462025-03-20T22:04:55.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53472025-03-20T22:04:55.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53482025-03-20T22:04:55.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53492025-03-20T22:04:55.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53502025-03-20T22:04:55.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53512025-03-20T22:04:55.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53522025-03-20T22:04:55.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53532025-03-20T22:04:55.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53542025-03-20T22:04:55.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53552025-03-20T22:04:55.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53562025-03-20T22:04:55.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
53572025-03-20T22:04:55.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53582025-03-20T22:04:55.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53592025-03-20T22:04:55.256ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
53602025-03-20T22:04:55.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.896542ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
53612025-03-20T22:04:55.267ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53622025-03-20T22:04:55.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53632025-03-20T22:04:55.296ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 59.602905ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53642025-03-20T22:04:55.412ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 13220e4d-691d-4f4e-a1eb-f5853445fee2 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53652025-03-20T22:04:55.412ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 13220e4d-691d-4f4e-a1eb-f5853445fee2 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53662025-03-20T22:04:55.414ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2098 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 13220e4d-691d-4f4e-a1eb-f5853445fee2 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53672025-03-20T22:04:56.433ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 580dd668-327a-4fa7-99be-717ed348b85c roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53682025-03-20T22:04:56.434ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 580dd668-327a-4fa7-99be-717ed348b85c resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53692025-03-20T22:04:56.435ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1935 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 580dd668-327a-4fa7-99be-717ed348b85c response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53702025-03-20T22:04:57.253ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 634 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 2266a1f5-ad78-4fd2-ac3c-d838090bafba response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
53712025-03-20T22:04:57.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = de7b1f1a-18aa-4a82-9d62-6b995d4fb683 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53722025-03-20T22:04:57.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = de7b1f1a-18aa-4a82-9d62-6b995d4fb683 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53732025-03-20T22:04:57.457ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1946 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = de7b1f1a-18aa-4a82-9d62-6b995d4fb683 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53742025-03-20T22:04:58.476ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7231f18c-2a0f-4f93-84e4-f1ef70d61e57 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53752025-03-20T22:04:58.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7231f18c-2a0f-4f93-84e4-f1ef70d61e57 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53762025-03-20T22:04:58.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1903 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7231f18c-2a0f-4f93-84e4-f1ef70d61e57 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53772025-03-20T22:04:59.499ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ad46e1e1-f226-4d6e-bc93-e8b7e7d77e4e roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53782025-03-20T22:04:59.500ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ad46e1e1-f226-4d6e-bc93-e8b7e7d77e4e resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53792025-03-20T22:04:59.501ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2547 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = ad46e1e1-f226-4d6e-bc93-e8b7e7d77e4e response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53802025-03-20T22:05:00.521ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fd2982a1-c663-4a9b-acbf-aac788b63e3d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53812025-03-20T22:05:00.522ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fd2982a1-c663-4a9b-acbf-aac788b63e3d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53822025-03-20T22:05:00.524ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2384 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = fd2982a1-c663-4a9b-acbf-aac788b63e3d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53832025-03-20T22:05:01.544ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4d1695db-845c-41bc-b094-ced8cfde844a roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53842025-03-20T22:05:01.544ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4d1695db-845c-41bc-b094-ced8cfde844a resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53852025-03-20T22:05:01.546ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2154 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 4d1695db-845c-41bc-b094-ced8cfde844a response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53862025-03-20T22:05:02.566ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 92f7e1b2-33de-475a-a1bb-68305f0361da roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53872025-03-20T22:05:02.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 92f7e1b2-33de-475a-a1bb-68305f0361da resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53882025-03-20T22:05:02.568ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2279 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 92f7e1b2-33de-475a-a1bb-68305f0361da response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53892025-03-20T22:05:03.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a87b61ed-0322-450c-9646-e8b2699beb7d roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53902025-03-20T22:05:03.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a87b61ed-0322-450c-9646-e8b2699beb7d resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53912025-03-20T22:05:03.590ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2497 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a87b61ed-0322-450c-9646-e8b2699beb7d response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53922025-03-20T22:05:04.610ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a52fb14b-95fe-428d-a054-c96747673dd7 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53932025-03-20T22:05:04.611ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a52fb14b-95fe-428d-a054-c96747673dd7 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53942025-03-20T22:05:04.613ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2742 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = a52fb14b-95fe-428d-a054-c96747673dd7 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
53952025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
53962025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
53972025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53982025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
53992025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
54002025-03-20T22:05:05.237ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
54012025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
54022025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54032025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
54042025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54052025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
54062025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54072025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
54082025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54092025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
54102025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
54112025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54122025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
54132025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
54142025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
54152025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54162025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
54172025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
54182025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
54192025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
54202025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
54212025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
54222025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Timeout
54232025-03-20T22:05:05.237ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
54242025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
54252025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
54262025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
54272025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
54282025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54292025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
54302025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
54312025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
54322025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
54332025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
54342025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
54352025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
54362025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
54372025-03-20T22:05:05.237ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f
54382025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
54392025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.391\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f"}
54402025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
54412025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
54422025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
54432025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54442025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
54452025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54462025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
54472025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
54482025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
54492025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54502025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
54512025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54522025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
54532025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
54542025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
54552025-03-20T22:05:05.237ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-20 21:55:05.237862816 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
54562025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
54572025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
54582025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
54592025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 29 reason = Timeout
54602025-03-20T22:05:05.237ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
54612025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54622025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54632025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54642025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
54652025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54662025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
54672025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
54682025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54692025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54702025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
54712025-03-20T22:05:05.238ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55400d62-c4af-483b-8223-1234d14cf0ad.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("d88b8f1a-4a6a-4e0b-ac74-40a39aba4124.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 3849, tv_nsec: 45502293 } })
54722025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
54732025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
54742025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
54752025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54762025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
54772025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
54782025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
54792025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54802025-03-20T22:05:05.238ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("58723511-3cb5-4c26-bebf-16620d9cd641.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("59acd018-3b3d-4cb6-bd0f-2dfaf2b8d965.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("64c794b9-d771-4c58-a8be-88acec6c4f47.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 3849, tv_nsec: 45654167 } })
54812025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
54822025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
54832025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
54842025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54852025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
54862025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
54872025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
54882025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
54892025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
54902025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.201962ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
54912025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
54922025-03-20T22:05:05.238ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.48984ms iteration = 15 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
54932025-03-20T22:05:05.239ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54942025-03-20T22:05:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54952025-03-20T22:05:05.240ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
54962025-03-20T22:05:05.240ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54972025-03-20T22:05:05.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
54982025-03-20T22:05:05.241ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.780561ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
54992025-03-20T22:05:05.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
55002025-03-20T22:05:05.243ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55012025-03-20T22:05:05.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55022025-03-20T22:05:05.245ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 7.805659ms iteration = 29 status = {"errors":[],"finish_invoked_ok":[]}
55032025-03-20T22:05:05.245ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 8.288662ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
55042025-03-20T22:05:05.246ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55052025-03-20T22:05:05.247ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55062025-03-20T22:05:05.247ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3849, tv_nsec: 55280725 } })
55072025-03-20T22:05:05.248ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3849, tv_nsec: 55499351 } })
55082025-03-20T22:05:05.248ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1171
55092025-03-20T22:05:05.248ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1171
55102025-03-20T22:05:05.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
55112025-03-20T22:05:05.250ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55122025-03-20T22:05:05.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
55132025-03-20T22:05:05.250ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
55142025-03-20T22:05:05.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55152025-03-20T22:05:05.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
55162025-03-20T22:05:05.260ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-20 21:55:05.237862816 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
55172025-03-20T22:05:05.260ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 23.040986ms iteration = 15 status = {"expiration":"2025-03-20T21:55:05.237862816Z","pruned":[]}
55182025-03-20T22:05:05.309ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55192025-03-20T22:05:05.310ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55202025-03-20T22:05:05.310ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55212025-03-20T22:05:05.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55222025-03-20T22:05:05.311ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55232025-03-20T22:05:05.313ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
55242025-03-20T22:05:05.313ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55252025-03-20T22:05:05.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55262025-03-20T22:05:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
55272025-03-20T22:05:05.314ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55282025-03-20T22:05:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55292025-03-20T22:05:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
55302025-03-20T22:05:05.315ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55312025-03-20T22:05:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55322025-03-20T22:05:05.318ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
55332025-03-20T22:05:05.318ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
55342025-03-20T22:05:05.318ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 80.876383ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
55352025-03-20T22:05:05.319ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55362025-03-20T22:05:05.319ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
55372025-03-20T22:05:05.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55382025-03-20T22:05:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
55392025-03-20T22:05:05.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55402025-03-20T22:05:05.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55412025-03-20T22:05:05.321ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55422025-03-20T22:05:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55432025-03-20T22:05:05.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55442025-03-20T22:05:05.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55452025-03-20T22:05:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55462025-03-20T22:05:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
55472025-03-20T22:05:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
55482025-03-20T22:05:05.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55492025-03-20T22:05:05.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
55502025-03-20T22:05:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
55512025-03-20T22:05:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55522025-03-20T22:05:05.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
55532025-03-20T22:05:05.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55542025-03-20T22:05:05.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55552025-03-20T22:05:05.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55562025-03-20T22:05:05.327ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55572025-03-20T22:05:05.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55582025-03-20T22:05:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55592025-03-20T22:05:05.329ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
55602025-03-20T22:05:05.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
55612025-03-20T22:05:05.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55622025-03-20T22:05:05.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55632025-03-20T22:05:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
55642025-03-20T22:05:05.335ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55652025-03-20T22:05:05.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55662025-03-20T22:05:05.337ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 99.692097ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
55672025-03-20T22:05:05.338ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55682025-03-20T22:05:05.339ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-317fd31f-9b13-492c-9d67-8468dff1ea05.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 3849, tv_nsec: 146475054 } })
55692025-03-20T22:05:05.339ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1171 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
55702025-03-20T22:05:05.340ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
55712025-03-20T22:05:05.340ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
55722025-03-20T22:05:05.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
55732025-03-20T22:05:05.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
55742025-03-20T22:05:05.342ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
55752025-03-20T22:05:05.342ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 104.686569ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
55762025-03-20T22:05:05.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
55772025-03-20T22:05:05.343ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55782025-03-20T22:05:05.343ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55792025-03-20T22:05:05.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55802025-03-20T22:05:05.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55812025-03-20T22:05:05.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55822025-03-20T22:05:05.348ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
55832025-03-20T22:05:05.351ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55842025-03-20T22:05:05.351ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 113.450734ms iteration = 29 status = {"errors":[],"requests_created_ok":[]}
55852025-03-20T22:05:05.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55862025-03-20T22:05:05.359ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
55872025-03-20T22:05:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
55882025-03-20T22:05:05.361ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
55892025-03-20T22:05:05.363ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
55902025-03-20T22:05:05.363ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55912025-03-20T22:05:05.364ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0eb4fefa-e389-4b31-bbbf-b85bd50a072f", "content-length": "26", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
55922025-03-20T22:05:05.364ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55932025-03-20T22:05:05.365ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
55942025-03-20T22:05:05.370ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55952025-03-20T22:05:05.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55962025-03-20T22:05:05.372ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55972025-03-20T22:05:05.372ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55982025-03-20T22:05:05.372ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
55992025-03-20T22:05:05.372ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56002025-03-20T22:05:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
56012025-03-20T22:05:05.374ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
56022025-03-20T22:05:05.374ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
56032025-03-20T22:05:05.375ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 137.401535ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
56042025-03-20T22:05:05.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56052025-03-20T22:05:05.375ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56062025-03-20T22:05:05.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56072025-03-20T22:05:05.377ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 139.967494ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56082025-03-20T22:05:05.378ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1179 zone_address = fd00:1122:3344:101::2
56092025-03-20T22:05:05.378ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56102025-03-20T22:05:05.379ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb23f239-2ddd-478c-939a-d09f0c2370a7", "content-length": "26", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
56112025-03-20T22:05:05.379ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1184 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56122025-03-20T22:05:05.381ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
56132025-03-20T22:05:05.381ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56142025-03-20T22:05:05.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56152025-03-20T22:05:05.386ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56162025-03-20T22:05:05.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56172025-03-20T22:05:05.391ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56182025-03-20T22:05:05.396ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 317fd31f-9b13-492c-9d67-8468dff1ea05, lookup_type: ById(317fd31f-9b13-492c-9d67-8468dff1ea05) } result = Ok(())
56192025-03-20T22:05:05.396ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56202025-03-20T22:05:05.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56212025-03-20T22:05:05.399ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 161.89384ms iteration = 29 status = 0
56222025-03-20T22:05:05.409ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
56232025-03-20T22:05:05.409ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
56242025-03-20T22:05:05.409ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
56252025-03-20T22:05:05.409ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56262025-03-20T22:05:05.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56272025-03-20T22:05:05.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 172.591487ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56282025-03-20T22:05:05.411ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56292025-03-20T22:05:05.412ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 174.658582ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
56302025-03-20T22:05:05.413ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 175.690599ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56312025-03-20T22:05:05.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56322025-03-20T22:05:05.416ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56332025-03-20T22:05:05.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
56342025-03-20T22:05:05.416ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56352025-03-20T22:05:05.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56362025-03-20T22:05:05.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56372025-03-20T22:05:05.417ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56382025-03-20T22:05:05.417ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
56392025-03-20T22:05:05.417ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 180.126408ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
56402025-03-20T22:05:05.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 675e983e-751a-42c7-ad22-bbd735b1e10f, lookup_type: ById(675e983e-751a-42c7-ad22-bbd735b1e10f) } result = Ok(())
56412025-03-20T22:05:05.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56422025-03-20T22:05:05.417ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56432025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56442025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56452025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
56462025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
56472025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
56482025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56492025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56502025-03-20T22:05:05.418ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
56512025-03-20T22:05:05.418ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 180.696053ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
56522025-03-20T22:05:05.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
56532025-03-20T22:05:05.420ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56542025-03-20T22:05:05.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56552025-03-20T22:05:05.421ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
56562025-03-20T22:05:05.421ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56572025-03-20T22:05:05.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56582025-03-20T22:05:05.422ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56592025-03-20T22:05:05.426ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
56602025-03-20T22:05:05.426ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56612025-03-20T22:05:05.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56622025-03-20T22:05:05.429ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f original_time_created = 2025-03-20 21:50:57.610361 UTC target_id = 675e983e-751a-42c7-ad22-bbd735b1e10f time_created = 2025-03-20 21:50:57.610361 UTC
56632025-03-20T22:05:05.429ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 192.352094ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"675e983e-751a-42c7-ad22-bbd735b1e10f","time_created":"2025-03-20T21:50:57.610361Z"}
56642025-03-20T22:05:05.458ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
56652025-03-20T22:05:05.458ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 221.252388ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
56662025-03-20T22:05:05.460ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
56672025-03-20T22:05:05.460ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 0 version = 2
56682025-03-20T22:05:05.460ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56692025-03-20T22:05:05.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56702025-03-20T22:05:05.467ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
56712025-03-20T22:05:05.467ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
56722025-03-20T22:05:05.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56732025-03-20T22:05:05.468ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56742025-03-20T22:05:05.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56752025-03-20T22:05:05.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56762025-03-20T22:05:05.473ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-20T21:51:01.993052Z, time_modified: 2025-03-20T21:51:01.993052Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4841a72c-7688-4028-b721-8d89119ea996, db_silo: Silo { identity: SiloIdentity { id: 4841a72c-7688-4028-b721-8d89119ea996, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-20T21:51:07.637647Z, time_modified: 2025-03-20T21:51:07.637647Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9" }] }) }
56772025-03-20T22:05:05.473ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 235.624692ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]}},"default_endpoint":{"silo_id":"4841a72c-7688-4028-b721-8d89119ea996","tls_certs":["6e847435719fde7a14769245ab887cf783e2c33700390f4e41086b70d0389fa9"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
56782025-03-20T22:05:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 288.403684ms iteration = 29 status = {}
56792025-03-20T22:05:05.526ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
56802025-03-20T22:05:05.550ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
56812025-03-20T22:05:05.550ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "fad966b0-a76c-4d11-af4e-33dddf967a92", "content-length": "2", "date": "Thu, 20 Mar 2025 22:05:04 GMT"} })
56822025-03-20T22:05:05.553ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
56832025-03-20T22:05:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
56842025-03-20T22:05:05.578ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 340.324153ms iteration = 32 status = {"physical_disks_added":0}
56852025-03-20T22:05:05.581ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
56862025-03-20T22:05:05.582ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "4b391ddd-7be5-452c-b37d-14569d85545f", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
56872025-03-20T22:05:05.582ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
56882025-03-20T22:05:05.584ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e5c1c669-aeee-4512-9b9e-c3aa33c967ab", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
56892025-03-20T22:05:05.584ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
56902025-03-20T22:05:05.586ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
56912025-03-20T22:05:05.586ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 348.693766ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
56922025-03-20T22:05:05.598ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1179 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 zone_address = fd00:1122:3344:101::2
56932025-03-20T22:05:05.598ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56942025-03-20T22:05:05.599ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d462e227-3845-4b7b-bca0-ecd273e3e78e", "content-length": "26", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
56952025-03-20T22:05:05.599ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1184 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56962025-03-20T22:05:05.599ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
56972025-03-20T22:05:05.599ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56982025-03-20T22:05:05.600ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1eb63956-2408-4316-a255-e5cd45dd1500", "content-length": "372", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
56992025-03-20T22:05:05.600ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57002025-03-20T22:05:05.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57012025-03-20T22:05:05.600ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1219 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
57022025-03-20T22:05:05.601ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57032025-03-20T22:05:05.602ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57042025-03-20T22:05:05.604ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 366.548324ms iteration = 29 status = null
57052025-03-20T22:05:05.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
57062025-03-20T22:05:05.605ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
57072025-03-20T22:05:05.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
57082025-03-20T22:05:05.605ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
57092025-03-20T22:05:05.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
57102025-03-20T22:05:05.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
57112025-03-20T22:05:05.608ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 825be6bf-d9ca-448e-adf9-43c7e20ec4ed dns_zone_name = oxide.test found_so_far = 1 version = 2
57122025-03-20T22:05:05.608ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
57132025-03-20T22:05:05.608ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-20 21:51:07.691085 UTC dns_group = external generation = 2
57142025-03-20T22:05:05.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 370.993603ms iteration = 15 status = {"generation":2}
57152025-03-20T22:05:05.609ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d348358b-e79e-46ce-aa00-8905c352aec1 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
57162025-03-20T22:05:05.609ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
57172025-03-20T22:05:05.609ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-20 21:51:06.492803 UTC dns_group = internal generation = 1
57182025-03-20T22:05:05.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 372.192055ms iteration = 15 status = {"generation":1}
57192025-03-20T22:05:05.616ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
57202025-03-20T22:05:05.616ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
57212025-03-20T22:05:05.616ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "71ceaced-48e6-4416-8d94-75a209ad13eb", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57222025-03-20T22:05:05.616ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57232025-03-20T22:05:05.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57242025-03-20T22:05:05.618ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57252025-03-20T22:05:05.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57262025-03-20T22:05:05.621ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57272025-03-20T22:05:05.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57282025-03-20T22:05:05.625ZINFOnexus (BackgroundTasks): VPC route manager sled 317fd31f-9b13-492c-9d67-8468dff1ea05 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
57292025-03-20T22:05:05.625ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57302025-03-20T22:05:05.625ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "58c5359b-3ab2-47d2-991a-9c1461f5ed5e", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57312025-03-20T22:05:05.627ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 389.763664ms iteration = 30 status = {}
57322025-03-20T22:05:05.633ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 484318eb-0684-4ed1-b6b7-3ea5f80d0ec6 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
57332025-03-20T22:05:05.633ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 484318eb-0684-4ed1-b6b7-3ea5f80d0ec6 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
57342025-03-20T22:05:05.635ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2553 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 484318eb-0684-4ed1-b6b7-3ea5f80d0ec6 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
57352025-03-20T22:05:05.657ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
57362025-03-20T22:05:05.657ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c78e022d-5439-41ea-8369-65dd637be639", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
57372025-03-20T22:05:05.657ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
57382025-03-20T22:05:05.659ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "6307e831-88c3-40e5-8410-446661cbcb4f", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
57392025-03-20T22:05:05.659ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
57402025-03-20T22:05:05.661ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "cb33e263-a1e6-4ab3-8ce6-568fa7816ab7", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
57412025-03-20T22:05:05.661ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
57422025-03-20T22:05:05.664ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
57432025-03-20T22:05:05.664ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 426.577889ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
57442025-03-20T22:05:05.710ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57452025-03-20T22:05:05.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57462025-03-20T22:05:05.712ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57472025-03-20T22:05:05.714ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57482025-03-20T22:05:05.714ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57492025-03-20T22:05:05.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57502025-03-20T22:05:05.716ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57512025-03-20T22:05:05.719ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57522025-03-20T22:05:05.719ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57532025-03-20T22:05:05.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57542025-03-20T22:05:05.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57552025-03-20T22:05:05.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57562025-03-20T22:05:05.735ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 17e96712-a76a-46ec-bf96-fec261aa8feb, rack_id: dc2d7387-2245-44cd-9115-6d413274bb18, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d) } rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
57572025-03-20T22:05:05.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57582025-03-20T22:05:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57592025-03-20T22:05:05.747ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
57602025-03-20T22:05:05.747ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5974443-849d-4635-be42-23010b09b61c", "content-length": "134", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57612025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
57622025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = d1ebbd6b-a00c-4d77-8a4d-29b8adfe9b9d
57632025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
57642025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
57652025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57662025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57672025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57682025-03-20T22:05:05.747ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57692025-03-20T22:05:05.748ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
57702025-03-20T22:05:05.748ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
57712025-03-20T22:05:05.748ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18
57722025-03-20T22:05:05.748ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57732025-03-20T22:05:05.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57742025-03-20T22:05:05.750ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
57752025-03-20T22:05:05.751ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7ebc936-6251-4b5e-9481-fcefce83a8d5", "content-length": "2", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57762025-03-20T22:05:05.751ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
57772025-03-20T22:05:05.751ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "eabf792d-c7d5-47c8-90f7-b40e363f0fcf", "content-length": "52", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57782025-03-20T22:05:05.751ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch_location = Switch0
57792025-03-20T22:05:05.751ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
57802025-03-20T22:05:05.834ZDEBGnexus (BackgroundTasks): client request SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
57812025-03-20T22:05:05.863ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6f1e62d6-d08e-4c90-9acc-8904cc066409", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57822025-03-20T22:05:05.863ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 switch = Switch0
57832025-03-20T22:05:05.863ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
57842025-03-20T22:05:05.863ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = dc2d7387-2245-44cd-9115-6d413274bb18 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "35423f20-5eae-4500-9038-264f91f851de", "content-length": "682", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57852025-03-20T22:05:05.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57862025-03-20T22:05:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57872025-03-20T22:05:05.864ZDEBGnexus (BackgroundTasks): client response SledAgent = 317fd31f-9b13-492c-9d67-8468dff1ea05 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "53f4c995-6d16-4cbe-8e6a-e2a2350a236b", "content-length": "691", "date": "Thu, 20 Mar 2025 22:05:05 GMT"} })
57882025-03-20T22:05:05.864ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
57892025-03-20T22:05:05.864ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
57902025-03-20T22:05:05.864ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
57912025-03-20T22:05:05.866ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 628.45475ms iteration = 30 status = {}
57922025-03-20T22:05:05.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57932025-03-20T22:05:05.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57942025-03-20T22:05:05.872ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57952025-03-20T22:05:05.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57962025-03-20T22:05:05.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57972025-03-20T22:05:05.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57982025-03-20T22:05:05.881ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 644.282373ms iteration = 29 status = {}
57992025-03-20T22:05:06.660ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f1c0a9e8-bcc0-4bd9-b591-6338738ffa84 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58002025-03-20T22:05:06.660ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f1c0a9e8-bcc0-4bd9-b591-6338738ffa84 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58012025-03-20T22:05:06.662ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2453 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = f1c0a9e8-bcc0-4bd9-b591-6338738ffa84 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58022025-03-20T22:05:07.252ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 525 local_addr = [fd00:1122:3344:101::c]:35583 method = GET remote_addr = [fd00:1122:3344:101::d]:41657 req_id = 3b1b1681-4406-4409-b09a-87c1ae57259d response_code = 200 uri = /8602d1a8-380a-4598-9f5c-9268e05f383a
58032025-03-20T22:05:07.680ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = eee9d822-4b6c-457c-97b9-ed81c093ac72 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58042025-03-20T22:05:07.680ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = eee9d822-4b6c-457c-97b9-ed81c093ac72 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58052025-03-20T22:05:07.683ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2433 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = eee9d822-4b6c-457c-97b9-ed81c093ac72 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58062025-03-20T22:05:08.702ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 61b235a1-b1e8-4f11-99e5-85a4429319a9 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58072025-03-20T22:05:08.702ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 61b235a1-b1e8-4f11-99e5-85a4429319a9 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58082025-03-20T22:05:08.704ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2519 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 61b235a1-b1e8-4f11-99e5-85a4429319a9 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58092025-03-20T22:05:09.724ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3ef94ba7-af37-415f-aad7-cd91e67a8615 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58102025-03-20T22:05:09.725ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3ef94ba7-af37-415f-aad7-cd91e67a8615 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58112025-03-20T22:05:09.727ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2661 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 3ef94ba7-af37-415f-aad7-cd91e67a8615 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58122025-03-20T22:05:10.750ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0acb6bbf-0387-432f-95f8-ef4aa51d6e7f roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58132025-03-20T22:05:10.750ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0acb6bbf-0387-432f-95f8-ef4aa51d6e7f resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58142025-03-20T22:05:10.752ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2348 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 0acb6bbf-0387-432f-95f8-ef4aa51d6e7f response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58152025-03-20T22:05:11.772ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 895aa52d-3820-4ccf-8842-f8eac138b5ea roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58162025-03-20T22:05:11.773ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 895aa52d-3820-4ccf-8842-f8eac138b5ea resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58172025-03-20T22:05:11.775ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2531 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 895aa52d-3820-4ccf-8842-f8eac138b5ea response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58182025-03-20T22:05:12.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7d9a99ca-8954-4aaf-bcf6-25bab1da42e1 roles = RoleSet { roles: {} } uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58192025-03-20T22:05:12.794ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7d9a99ca-8954-4aaf-bcf6-25bab1da42e1 resource = Database result = Ok(()) uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05
58202025-03-20T22:05:12.796ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:64383 req_id = 7d9a99ca-8954-4aaf-bcf6-25bab1da42e1 response_code = 200 uri = /probes/317fd31f-9b13-492c-9d67-8468dff1ea05